Preserving differential privacy in convolutional deep belief networks
Open accessThe DOI matches its Crossref record; 19 of the 19 references it cites were found in a registry or an index.
Abstract
The remarkable development of deep learning in medicine and healthcare domain presents obvious privacy issues, when deep neural networks are built on users’ personal and highly sensitive data, e.g., clinical records, user profiles, biomedical images, etc. However, only a few scientific studies on preserving privacy in deep learning have been conducted. In this paper, we focus on developing a private convolutional deep belief network (pCDBN), which essentially is a convolutional deep belief network (CDBN) under differential privacy. Our main idea of enforcing ϵ\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\epsilon $$\end{document}-differential privacy is to leverage the functional mechanism to perturb the energy-based objective functions of traditional CDBNs, rather than their results. One key contribution of this work is that we propose the use of Chebyshev expansion to derive the approximate polynomial representation of objective functions. Our theoretical analysis shows that we can further derive the sensitivity and error bounds of the approximate polynomial representation. As a result, preserving differential privacy in CDBNs is feasible. We applied our model in a health social network, i.e., YesiWell data, and in a handwriting digit dataset, i.e., MNIST data, for human behavior prediction, human behavior classification, and handwriting digit recognition tasks. Theoretical analysis and rigorous experimental evaluations show that the pCDBN is highly effective. It significantly outperforms existing solutions.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805
Human agency in social cognitive theory.
Bandura, A. (1989) · American Psychologist · doi:10.1037/0003-066x.44.9.1175 · Cited by 8,263 · cited 2×
“To model this effect (i.e., also considered as a form of self-motivation in social cognitive theory [8]), we first aggregate his personal attributes in the last N timestamps into a d × N matrix, which will be considered the visible input V .”
BackgroundMethodCalibrating Noise to Sensitivity in Private Data Analysis
Dwork, C., McSherry, F., Nissim, K., & Smith, A. (2006) · Lecture Notes in Computer Science · doi:10.1007/11681878_14 · Cited by 7,272
Training Products of Experts by Minimizing Contrastive Divergence
Hinton, G. E. (2002) · Neural Computation · doi:10.1162/089976602760128018 · Cited by 4,879 · cited 2×
“In our work, contrastive divergent algorithm [30] was used to optimize the energy function, and back-propagation was used to optimize the cross-entropy error function in the softmax layer.”
BackgroundMethodGreedy layer-wise training of deep networks
Bengio, Y., Lamblin, P., Popovici, D., & Larochelle, H. (2007) · Advances in Neural Information Processing Systems 19 · doi:10.7551/mitpress/7503.003.0024 · Cited by 4,841
On Pixel-Wise Explanations for Non-Linear Classifier Decisions by Layer-Wise Relevance Propagation
Bach, S., Binder, A., Montavon, G., Klauschen, F., Müller, K.-R., & Samek, W. (2015) · PLOS ONE · doi:10.1371/journal.pone.0130140 · Cited by 4,524 · cited 1×
“Similar to Layer-wise Relevance Propagation [7], the approximation errors will propagate across neural layers.”
ResultLee, H., Grosse, R., Ranganath, R., & Ng, A. Y. (2009) · Proceedings of the 26th Annual International Conference on Machine Learning · doi:10.1145/1553374.1553453 · Cited by 2,412
Cited by
Differentially Private Generative Adversarial Network
Xie, L., Lin, K., Wang, S., Wang, F., & Zhou, J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.06739 · Cited by 295
Privacy and Security Issues in Deep Learning: A Survey
Liu, X., Xie, L., Wang, Y., Zou, J., Xiong, J., Ying, Z., & Vasilakos, A. V. (2021) · IEEE Access · doi:10.1109/access.2020.3045078 · Cited by 282
Evaluating Differentially Private Machine Learning in Practice
Jayaraman, B., & Evans, D. (2019) · arXiv · doi:10.48550/arxiv.1902.08874 · Cited by 265
Yang, Q., Liu, Y., Cheng, Y., Kang, Y., Chen, T., & Yu, H. (2019) · Synthesis lectures on artificial intelligence and machine learning · doi:10.2200/s00960ed2v01y201910aim043 · Cited by 261
A review of privacy-preserving techniques for deep learning
Boulemtafes, A., Derhab, A., & Challal, Y. (2020) · Neurocomputing · doi:10.1016/j.neucom.2019.11.041 · Cited by 211
“Based on the same idea of perturbing objective functions, Phan et al. [34] described another approach for releasing a private Convolutional Deep Belief Network (CDBN).”
BackgroundMethodInfluential
Metadata
- Authors
- Phan, N., Wu, X., & Dou, D.
- Venue
- Machine Learning
- Published
- 2017-07-13
- Volume
- 106
- Issue
- 9-10
- Pages
- 1681-1704
- ISSN
- 0885-6125 (ISSN-L)
- 1573-0565
- OpenAlex
- W2729742878
- Cited by
- 95
- References
- 19
- FWCI
- 5.90
- Type
- journal article
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Preserving differential privacy in convolutional deep belief networks. https://app.cochat.ai/papers/10.1007/s10994-017-5656-2
- The paper
- Phan, N., Wu, X., & Dou, D. (2017). Preserving differential privacy in convolutional deep belief networks. Machine Learning. https://doi.org/10.1007/s10994-017-5656-2
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.