CoChat
Sign Up
Cited by 156Digital Media Forensic DetectionAdvanced Steganography and Watermarking TechniquesPrivacy-Preserving Technologies in Data

The DOI matches its Crossref record; 33 of the 36 references it cites were found in a registry or an index.

Learning to Break Deep Perceptual Hashing: The Use Case NeuralHash

Open access
Struppek, Lukas, Hintersdorf, Dominik, Neider, Daniel, Kersting, Kristian·2022·2022 ACM Conference on Fairness Accountability and Transparency
DOI 10.1145/3531146.3533073
Open this paper in CoChat — free account

Abstract

Apple recently revealed its deep perceptual hashing system NeuralHash to detect child sexual abuse material (CSAM) on user devices before files are uploaded to its iCloud service. Public criticism quickly arose regarding the protection of user privacy and the system’s reliability. In this paper, we present the first comprehensive empirical analysis of deep perceptual hashing based on NeuralHash. Specifically, we show that current deep perceptual hashing may not be robust. An adversary can manipulate the hash values by applying slight changes in images, either induced by gradient-based approaches or simply by performing standard image transformations, forcing or preventing hash collisions. Such attacks permit malicious actors easily to exploit the detection system: from hiding abusive material to framing innocent users, everything is possible. Moreover, using the hash values, inferences can still be made about the data stored on user devices. In our view, based on our results, deep perceptual hashing in its current form is generally not ready for robust client-side scanning and should not be used from a privacy perspective.1

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1145/3531146.3533073

Cites

33 of 33 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Flagged first, then by citations

Show all 36 references

3 of the 36 references couldn’t be matched to a known paper.

Research works printed but not linked: 22

  1. Hal Abelson Ross Anderson Steven M. Bellovin Josh Benaloh Matt Blaze Jon Callas Whitfield Diffie Susan Landau Peter G. Neumann Ronald L. Rivest Jeffrey I. Schiller Bruce Schneier Vanessa Teague and Carmela Troncoso. 2021. Bugs in our Pockets: The Risks of Client-Side Scanning. CoRR abs/2110.07450(2021). · arXiv
  2. Mathias Appel. 2017. PIXNIO-226632-4681x3100.jpeg. https://pixnio.com/creative-commons-license, accessed Sept. 23, 2021; Cropped; License: https://pixnio.com/creative-commons-license.
  3. Mihir Bellare. 2021. The Apple PSI Protocol. https://www.apple.com/child-safety/pdf/Technical_Assessment_of_CSAM_Detection_Mihir_Bellare.pdf, accessed: Oct. 1, 2021.
  4. Patrick Breyer, Alviina Alametsä, Rosa D’Amato, Pernando Barrena, Saskia Bricmont, Antoni Comín, Gwendoline Delbos-Corfield, Francesca Donato, Cornelia Ernst, Claudia Gamon, Markéta Gregorová, Francisco Guerreiro, Svenja Hahn, Irena Joveva, Petra Kammerevert, Marcel Kolaja, Moritz Körner, Karen Melchior, Clara Ponsatí, and Mikuláš Peksa. 2021. Cross-Party Letter of Member of the European Parliament Against General Monitoring. https://www.patrick-breyer.de/wp-content/uploads/2021/11/20211020_Letter_General_Monitoring.pdf, accessed: Nov. 19, 2021.
  5. Francesco Croce and Matthias Hein. 2020. Minimally distorted Adversarial Examples with a Fast Adaptive Boundary Attack. In Proceedings of the 37th International Conference on Machine Learning (ICML)(Proceedings of Machine Learning Research, Vol. 119). 2196–2205.
  6. Xiaoyi Dong Dongdong Chen Jianmin Bao Chuan Qin Lu Yuan Weiming Zhang Nenghai Yu and Dong Chen. 2020. GreedyFool: Distortion-Aware Sparse Adversarial Attack. In Conference on Neural Information Processing Systems (NeurIPS) Hugo Larochelle Marc’Aurelio Ranzato Raia Hadsell Maria-Florina Balcan and Hsuan-Tien Lin (Eds.).
  7. Yinpeng Dong, Fangzhou Liao, Tianyu Pang, Hang Su, Jun Zhu, Xiaolin Hu, and Jianguo Li. 2018. Boosting Adversarial Attacks With Momentum. In 2018 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 9185–9193.
  8. Noam Eshed. 2020. Novelty detection and analysis in convolutional neural networks. Master’s thesis. Cornell University.
  9. Facebook. 2019. Open-Sourcing Photo- and Video-Matching Technology to Make the Internet Safer. https://about.fb.com/news/2019/08/open-source-photo-video-matching, accessed Oct. 18, 2021.
  10. National Center for Missing & Exploited Children. 2021. CyberTipline - By the Numbers. https://www.missingkids.org/gethelpnow/cybertipline, accessed: Oct. 1, 2021.
  11. David Forsyth. 2021. Apple’s CSAM detection technology. https://www.apple.com/child-safety/pdf/Technical_Assessment_of_CSAM_Detection_David_Forsyth.pdf, accessed: Oct. 1, 2021.
  12. Ian Goodfellow Jonathon Shlens and Christian Szegedy. 2015. Explaining and Harnessing Adversarial Examples. In ICLR.
  13. Apple Inc.2021. CSAM Detection - Technical Summary. https://www.apple.com/child-safety/pdf/CSAM_Detection_Technical_Summary.pdf, accessed: Sept. 22, 2021.
  14. Aditya Khosla, Nityananda Jayadevaprakash, Bangpeng Yao, and Li Fei-Fei. 2011. Novel Dataset for Fine-Grained Image Categorization. In First Workshop on Fine-Grained Visual Categorization, CVPR.
  15. Aleksander Madry Aleksandar Makelov Ludwig Schmidt Dimitris Tsipras and Adrian Vladu. 2018. Towards Deep Learning Models Resistant to Adversarial Attacks. In ICLR.
  16. Microsoft. 2015. PhotoDNA. https://www.microsoft.com/en-us/photodna, accessed Oct. 18, 2021.
  17. Official Journal of the European Union. 2021. Regulation (EU) 2021/1232. https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32021R1232, accessed: Nov. 16, 2021.
  18. Nicolas Papernot, Patrick D. McDaniel, Somesh Jha, Matt Fredrikson, Z. Berkay Celik, and Ananthram Swami. 2016. The Limitations of Deep Learning in Adversarial Settings. In IEEE European Symposium on Security and Privacy (EuroS&P). 372–387.
  19. Benny Pinkas. 2021. A Review of the Cryptography Behind the Apple PSI System. https://www.apple.com/child-safety/pdf/Technical_Assessment_of_CSAM_Detection_Benny_Pinkas.pdf, accessed: Oct. 1, 2021.
  20. Christian Szegedy Wojciech Zaremba Ilya Sutskever Joan Bruna Dumitru Erhan Ian Goodfellow and Rob Fergus. 2014. Intriguing properties of neural networks. In ICLR.
  21. F. Zhao Yongzhen Huang Liang Wang and Tieniu Tan. 2015. Deep semantic ranking based hashing for multi-label image retrieval. In CVPR. 1556–1564.
  22. Mingkang Zhu, Tianlong Chen, and Zhangyang Wang. 2021. Sparse and Imperceptible Adversarial Attack via a Homotopy Algorithm. In Proceedings of the 38th International Conference on Machine Learning (ICML), Marina Meila and Tong Zhang (Eds.). Vol. 139. 12868–12877.

Cited by

Sorted by citations

Items 101–156 of 156

Related

See all

Metadata

Authors
Struppek, L., Hintersdorf, D., Neider, D., & Kersting, K.
Affiliations
  • Technische Universität Darmstadt (DE)
  • Carl von Ossietzky Universität Oldenburg (DE)
  • Department of Computer Science and Centre for Cognitive Science, Technical University of Darmstadt, Germany and Hessian Center for Artificial Intelligence, Germany
ORCID
Venue
2022 ACM Conference on Fairness Accountability and Transparency
Publisher
ACM
Published
2022-06-20
Pages
58-69
DOI
10.1145/3531146.3533073
OpenAlex
W3214430180
Cited by
153
References
36
Citations by year
2026: 12 · 2025: 11 · 2024: 14 · 2023: 112 · 2022: 4
FWCI
12.78
Type
conference paper
Language
en
Open access
Licence
Locations
  • ACM Conference on Fairness, Accountability, and Transparency (FAccT) (publishedVersion, cc-by-sa)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Funders
  • Bundesministerium für Bildung und Forschung (13N15343)
  • National Research Center for Applied Cybersecurity ATHENE
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Digital Media Forensic Detection
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Steganography and Watermarking Techniques
  • Physical Sciences › Computer Science › Artificial Intelligence › Privacy-Preserving Technologies in Data
Keywords
hash collision attacks, adversarial attacks, privacy, iCloud, image editing, hash robustness

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review