The DOI matches its Crossref record; 33 of the 36 references it cites were found in a registry or an index.
Learning to Break Deep Perceptual Hashing: The Use Case NeuralHash
Open accessAbstract
Apple recently revealed its deep perceptual hashing system NeuralHash to detect child sexual abuse material (CSAM) on user devices before files are uploaded to its iCloud service. Public criticism quickly arose regarding the protection of user privacy and the system’s reliability. In this paper, we present the first comprehensive empirical analysis of deep perceptual hashing based on NeuralHash. Specifically, we show that current deep perceptual hashing may not be robust. An adversary can manipulate the hash values by applying slight changes in images, either induced by gradient-based approaches or simply by performing standard image transformations, forcing or preventing hash collisions. Such attacks permit malicious actors easily to exploit the detection system: from hiding abusive material to framing innocent users, everything is possible. Moreover, using the hash values, inferences can still be made about the data stored on user devices. In our view, based on our results, deep perceptual hashing in its current form is generally not ready for robust client-side scanning and should not be used from a privacy perspective.1
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Surprisingly, the newest edition of NeuralHash that we experimented with was still not robust to filtering, rotating, and mirroring, similar to the first edition experimented in [11][16].”
From Robustness of Practical Perceptual Hashing Algorithms to Hash-Evasion and Hash-Inversion Attacks (2024) · cites this paper as [16]
Experimental — accuracy not yet measuredThe check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“- PhotoDNA is not robust to: filtering, rotating, resizing, mirroring, bordering, cropping;”
From Robustness of Practical Perceptual Hashing Algorithms to Hash-Evasion and Hash-Inversion Attacks (2024) · cites this paper as [16]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
Image quality assessment: from error visibility to structural similarity
Wang, Z., Bovik, A. C., Sheikh, H. R., & Simoncelli, E. P. (2004) · IEEE Transactions on Image Processing · doi:10.1109/tip.2003.819861 · Cited by 57,892
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380
PyTorch: An Imperative Style, High-Performance Deep Learning Library
Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L., et al. (2019) · arXiv · doi:10.48550/arxiv.1912.01703 · Cited by 31,473
A Computational Approach to Edge Detection
Canny, J. (1986) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.1986.4767851 · Cited by 30,004
GAN（Generative Adversarial Nets）
淳司, 柴. (2017) · Journal of Japan Society for Fuzzy Theory and Intelligent Informatics · doi:10.3156/jsoft.29.5_177_2 · Cited by 20,507
The Unreasonable Effectiveness of Deep Features as a Perceptual Metric
Zhang, R., Isola, P., Efros, A. A., Shechtman, E., & Wang, O. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00068 · Cited by 14,623
A Style-Based Generator Architecture for Generative Adversarial Networks
Karras, T., Laine, S., & Aila, T. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00453 · Cited by 11,714
Howard, A., Sandler, M., Chen, B., Wang, W., Chen, L.-C., Tan, M., Chu, G., Vasudevan, V., Zhu, Y., Pang, R., et al. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00140 · Cited by 9,708
Towards Evaluating the Robustness of Neural Networks
Carlini, N., & Wagner, D. (2017) · 2017 IEEE Symposium on Security and Privacy (SP) · doi:10.1109/sp.2017.49 · Cited by 8,859
Analyzing and Improving the Image Quality of StyleGAN
Karras, T., Laine, S., Aittala, M., Hellsten, J., Lehtinen, J., & Aila, T. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00813 · Cited by 6,332
Dimensionality Reduction by Learning an Invariant Mapping
Hadsell, R., Chopra, S., & LeCun, Y. (2006) · 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Volume 2 (CVPR'06) · doi:10.1109/cvpr.2006.100 · Cited by 5,336
Approximate nearest neighbors: towards removing the curse of dimensionality
Indyk, P., & Motwani, R. (1998) · Proceedings of the thirtieth annual ACM symposium on Theory of computing - STOC '98 · doi:10.1145/276698.276876 · Cited by 4,188
(1999) · Cited by 3,186
One Pixel Attack for Fooling Deep Neural Networks
Su, J., Vargas, D. V., & Sakurai, K. (2019) · IEEE Transactions on Evolutionary Computation · doi:10.1109/tevc.2019.2890858 · Cited by 2,406
StarGAN v2: Diverse Image Synthesis for Multiple Domains
Choi, Y., Uh, Y., Yoo, J., & Ha, J.-W. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00821 · Cited by 1,663
Training Generative Adversarial Networks with Limited Data
Karras, T., Aittala, M., Hellsten, J., Laine, S., Lehtinen, J., & Aila, T. (2020) · arXiv · doi:10.48550/arxiv.2006.06676 · Cited by 1,170
Deep Supervised Hashing for Fast Image Retrieval
Liu, H., Wang, R., Shan, S., & Chen, X. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.227 · Cited by 933
Deep hashing for compact binary codes learning
Liong, V. E., Lu, J., Wang, G., Moulin, P., & Zhou, J. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298862 · Cited by 545
BreCaHAD: a dataset for breast cancer histopathological annotation and diagnosis
Aksaç, A., Demetrick, D. J., Özyer, T., & Alhajj, R. (2019) · BMC Research Notes · doi:10.1186/s13104-019-4121-7 · Cited by 123
Adversarial Examples for Hamming Space Search
Yang, E., Liu, T., Deng, C. D., & Tao, D. (2018) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2018.2882908 · Cited by 121
Targeted Attack for Deep Hashing Based Retrieval
Bai, J., Chen, B., Li, Y., Wu, D., Guo, W., Xia, S., & Yang, E. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58452-8_36 · Cited by 90
Deep Supervised Hashing for Multi-Label and Large-Scale Image Retrieval
Wu, D., Lin, Z., Li, B., Ye, M., & Wang, W. (2017) · International Conference on Multimedia Retrieval · doi:10.1145/3078971.3078989 · Cited by 66
Prototype-supervised Adversarial Network for Targeted Attack of Deep Hashing
Wang, X., Zhang, Z., Wu, B., Shen, F., & Lu, G. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.01609 · Cited by 57
It's Not What It Looks Like: Manipulating Perceptual Hashing based Applications
Hao, Q., Luo, L., Jan, S. T. K., & Wang, G. (2021) · ACM SIGSAC Conference on Computer and Communications Security (CCS) · doi:10.1145/3460120.3484559 · Cited by 43
Xiao, Y., & Wang, C. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00197 · Cited by 37
Evaluating robustness of perceptual image hashing algorithms
Drmic, A., Silic, M., Delac, G., Vladimir, K., & Kurdija, A. S. (2017) · 2017 40th International Convention on Information and Communication Technology, Electronics and Microelectronics (MIPRO) · doi:10.23919/mipro.2017.7973569 · Cited by 35
Targeted Attack and Defense for Deep Hashing
Wang, X., Zhang, Z., Lu, G., & Xu, Y. D. (2021) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3404835.3463233 · Cited by 26
Jain, S., Cretu, A.-M., & de Montjoye, Y.-A. (2021) · arXiv · doi:10.48550/arxiv.2106.09820 · Cited by 21 · cited 1×
“Similar works investigated the robustness of non-deep perceptual hashing algorithms against adversarial attacks [24, 28] and their robustness [15] against visible image modifications.”
BackgroundRevHashNet: Perceptually de-hashing real-valued image hashes for similarity retrieval
Wang, Y., Palangi, H., Wang, Z. J., & Wang, H. (2018) · Signal Processing Image Communication · doi:10.1016/j.image.2018.06.018 · Cited by 17
Adversarial collision attacks on image hashing functions
Dolhansky, B., & Ferrer, C. C. (2020) · arXiv · doi:10.48550/arxiv.2011.09473 · Cited by 13
7 The United Kingdom Government
McFadden, J., & McFadzean, D. (2016) · Edinburgh University Press eBooks · doi:10.1515/9780748698318-010 · Cited by 6
Research works printed but not linked: 22
- Hal Abelson Ross Anderson Steven M. Bellovin Josh Benaloh Matt Blaze Jon Callas Whitfield Diffie Susan Landau Peter G. Neumann Ronald L. Rivest Jeffrey I. Schiller Bruce Schneier Vanessa Teague and Carmela Troncoso. 2021. Bugs in our Pockets: The Risks of Client-Side Scanning. CoRR abs/2110.07450(2021). · arXiv
- Mathias Appel. 2017. PIXNIO-226632-4681x3100.jpeg. https://pixnio.com/creative-commons-license, accessed Sept. 23, 2021; Cropped; License: https://pixnio.com/creative-commons-license.
- Mihir Bellare. 2021. The Apple PSI Protocol. https://www.apple.com/child-safety/pdf/Technical_Assessment_of_CSAM_Detection_Mihir_Bellare.pdf, accessed: Oct. 1, 2021.
- Patrick Breyer, Alviina Alametsä, Rosa D’Amato, Pernando Barrena, Saskia Bricmont, Antoni Comín, Gwendoline Delbos-Corfield, Francesca Donato, Cornelia Ernst, Claudia Gamon, Markéta Gregorová, Francisco Guerreiro, Svenja Hahn, Irena Joveva, Petra Kammerevert, Marcel Kolaja, Moritz Körner, Karen Melchior, Clara Ponsatí, and Mikuláš Peksa. 2021. Cross-Party Letter of Member of the European Parliament Against General Monitoring. https://www.patrick-breyer.de/wp-content/uploads/2021/11/20211020_Letter_General_Monitoring.pdf, accessed: Nov. 19, 2021.
- Francesco Croce and Matthias Hein. 2020. Minimally distorted Adversarial Examples with a Fast Adaptive Boundary Attack. In Proceedings of the 37th International Conference on Machine Learning (ICML)(Proceedings of Machine Learning Research, Vol. 119). 2196–2205.
- Xiaoyi Dong Dongdong Chen Jianmin Bao Chuan Qin Lu Yuan Weiming Zhang Nenghai Yu and Dong Chen. 2020. GreedyFool: Distortion-Aware Sparse Adversarial Attack. In Conference on Neural Information Processing Systems (NeurIPS) Hugo Larochelle Marc’Aurelio Ranzato Raia Hadsell Maria-Florina Balcan and Hsuan-Tien Lin (Eds.).
- Yinpeng Dong, Fangzhou Liao, Tianyu Pang, Hang Su, Jun Zhu, Xiaolin Hu, and Jianguo Li. 2018. Boosting Adversarial Attacks With Momentum. In 2018 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 9185–9193.
- Noam Eshed. 2020. Novelty detection and analysis in convolutional neural networks. Master’s thesis. Cornell University.
- Facebook. 2019. Open-Sourcing Photo- and Video-Matching Technology to Make the Internet Safer. https://about.fb.com/news/2019/08/open-source-photo-video-matching, accessed Oct. 18, 2021.
- National Center for Missing & Exploited Children. 2021. CyberTipline - By the Numbers. https://www.missingkids.org/gethelpnow/cybertipline, accessed: Oct. 1, 2021.
- David Forsyth. 2021. Apple’s CSAM detection technology. https://www.apple.com/child-safety/pdf/Technical_Assessment_of_CSAM_Detection_David_Forsyth.pdf, accessed: Oct. 1, 2021.
- Ian Goodfellow Jonathon Shlens and Christian Szegedy. 2015. Explaining and Harnessing Adversarial Examples. In ICLR.
- Apple Inc.2021. CSAM Detection - Technical Summary. https://www.apple.com/child-safety/pdf/CSAM_Detection_Technical_Summary.pdf, accessed: Sept. 22, 2021.
- Aditya Khosla, Nityananda Jayadevaprakash, Bangpeng Yao, and Li Fei-Fei. 2011. Novel Dataset for Fine-Grained Image Categorization. In First Workshop on Fine-Grained Visual Categorization, CVPR.
- Aleksander Madry Aleksandar Makelov Ludwig Schmidt Dimitris Tsipras and Adrian Vladu. 2018. Towards Deep Learning Models Resistant to Adversarial Attacks. In ICLR.
- Microsoft. 2015. PhotoDNA. https://www.microsoft.com/en-us/photodna, accessed Oct. 18, 2021.
- Official Journal of the European Union. 2021. Regulation (EU) 2021/1232. https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32021R1232, accessed: Nov. 16, 2021.
- Nicolas Papernot, Patrick D. McDaniel, Somesh Jha, Matt Fredrikson, Z. Berkay Celik, and Ananthram Swami. 2016. The Limitations of Deep Learning in Adversarial Settings. In IEEE European Symposium on Security and Privacy (EuroS&P). 372–387.
- Benny Pinkas. 2021. A Review of the Cryptography Behind the Apple PSI System. https://www.apple.com/child-safety/pdf/Technical_Assessment_of_CSAM_Detection_Benny_Pinkas.pdf, accessed: Oct. 1, 2021.
- Christian Szegedy Wojciech Zaremba Ilya Sutskever Joan Bruna Dumitru Erhan Ian Goodfellow and Rob Fergus. 2014. Intriguing properties of neural networks. In ICLR.
- F. Zhao Yongzhen Huang Liang Wang and Tieniu Tan. 2015. Deep semantic ranking based hashing for multi-label image retrieval. In CVPR. 1556–1564.
- Mingkang Zhu, Tianlong Chen, and Zhangyang Wang. 2021. Sparse and Imperceptible Adversarial Attack via a Homotopy Algorithm. In Proceedings of the 38th International Conference on Machine Learning (ICML), Marina Meila and Tong Zhang (Eds.). Vol. 139. 12868–12877.
Cited by
Sorted by citations
Poisoning Web-Scale Training Datasets is Practical
Carlini, N., Jagielski, M., Choquette-Choo, C. A., Paleka, D., Pearce, W., Anderson, H., Terzis, A., Thomas, K., & Tramèr, F. (2024) · 2024 IEEE Symposium on Security and Privacy (SP) · doi:10.1109/sp54263.2024.00179 · Cited by 138
Poisoning Web-Scale Training Datasets is Practical
Carlini, N., Jagielski, M., Choquette-Choo, C. A., Paleka, D., Pearce, W., Anderson, H., Terzis, A., Thomas, K., & Tramèr, F. (2023) · arXiv · doi:10.48550/arxiv.2302.10149 · Cited by 138
“Switching to a perceptual hash function (which aims for invariance to small image changes) would lead to higher utility, but would not meaningfully prevent our poisoning attacks because an attacker could upload poisoned images that were adversarially modified to fool the perceptual hash [26,29,68].”
BackgroundExploiting Cultural Biases via Homoglyphs in Text-to-Image Synthesis
Struppek, L., Hintersdorf, D., Friedrich, F., Br, M., Schramowski, P., & Kersting, K. (2023) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.15388 · Cited by 77
Self-citationSoK: Content Moderation for End-to-End Encryption
Scheffler, S., & Mayer, J. R. (2023) · Proceedings on Privacy Enhancing Technologies · doi:10.56553/popets-2023-0060 · Cited by 43
Hintersdorf, D., Struppek, L., Brack, M., Friedrich, F., Schramowski, P., & Kersting, K. (2024) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.15461 · Cited by 27
Self-citation
Metadata
- Authors
- Struppek, L., Hintersdorf, D., Neider, D., & Kersting, K.
- Affiliations
- Technische Universität Darmstadt (DE)
- Carl von Ossietzky Universität Oldenburg (DE)
- Department of Computer Science and Centre for Cognitive Science, Technical University of Darmstadt, Germany and Hessian Center for Artificial Intelligence, Germany
- Venue
- 2022 ACM Conference on Fairness Accountability and Transparency
- Publisher
- ACM
- Published
- 2022-06-20
- Pages
- 58-69
- OpenAlex
- W3214430180
- Cited by
- 153
- References
- 36
- Citations by year
- 2026: 12 · 2025: 11 · 2024: 14 · 2023: 112 · 2022: 4
- FWCI
- 12.78
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Locations
- ACM Conference on Fairness, Accountability, and Transparency (FAccT) (publishedVersion, cc-by-sa)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- Bundesministerium für Bildung und Forschung (13N15343)
- National Research Center for Applied Cybersecurity ATHENE
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Digital Media Forensic Detection
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Steganography and Watermarking Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Privacy-Preserving Technologies in Data
- Keywords
- hash collision attacks, adversarial attacks, privacy, iCloud, image editing, hash robustness
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
