Cited by 155Digital Media Forensic DetectionAdvanced Steganography and Watermarking TechniquesPrivacy-Preserving Technologies in Data

Learning to Break Deep Perceptual Hashing: The Use Case NeuralHash

Open access
Struppek, Lukas, Hintersdorf, Dominik, Neider, Daniel, Kersting, Kristian·2022·2022 ACM Conference on Fairness Accountability and Transparency

Abstract

Apple recently revealed its deep perceptual hashing system NeuralHash to detect child sexual abuse material (CSAM) on user devices before files are uploaded to its iCloud service. Public criticism quickly arose regarding the protection of user privacy and the system’s reliability. In this paper, we present the first comprehensive empirical analysis of deep perceptual hashing based on NeuralHash. Specifically, we show that current deep perceptual hashing may not be robust. An adversary can manipulate the hash values by applying slight changes in images, either induced by gradient-based approaches or simply by performing standard image transformations, forcing or preventing hash collisions. Such attacks permit malicious actors easily to exploit the detection system: from hiding abusive material to framing innocent users, everything is possible. Moreover, using the hash values, inferences can still be made about the data stored on user devices. In our view, based on our results, deep perceptual hashing in its current form is generally not ready for robust client-side scanning and should not be used from a privacy perspective.1

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1145/3531146.3533073

Cites

33 of 33 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • The newest edition of NeuralHash that we experimented with was still not robust to filtering, rotating, and mirroring, similar to the first edition experimented in [11][16].

    “Surprisingly, the newest edition of NeuralHash that we experimented with was still not robust to filtering, rotating, and mirroring, similar to the first edition experimented in [11][16].”

    From Robustness of Practical Perceptual Hashing Algorithms to Hash-Evasion and Hash-Inversion Attacks (2024) · cites this paper as [16]

    Section 5 (pp. 8–9) reports that the NeuralHash version studied was susceptible to rotation and image flipping (mirroring). It also tests color and contrast changes, but does not establish a general lack of robustness to filtering. The claim about the newest edition is the citing authors’ own result, not a finding attributed to this source. (LLM, conf=0.83)

    Experimental — accuracy not yet measured
  • PhotoDNA is not robust to filtering, rotating, resizing, mirroring, bordering, and cropping.

    “- PhotoDNA is not robust to: filtering, rotating, resizing, mirroring, bordering, cropping;”

    From Robustness of Practical Perceptual Hashing Algorithms to Hash-Evasion and Hash-Inversion Attacks (2024) · cites this paper as [16]

    Section 5 (pp. 8–9) tests these kinds of transformations on NeuralHash, not PhotoDNA. The passages mentioning PhotoDNA in Sections 2.1 (p. 2) and 7 (p. 11) do not address its robustness to filtering, rotation, resizing, mirroring, borders, or cropping. A search for PhotoDNA and transformation robustness found no passage establishing the attributed claim. (LLM, conf=0.77)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

3 of the 36 references couldn’t be matched to a known paper.

Research works printed but not linked: 10

  1. [1][1] Hal Abelson, Ross Anderson, Steven M. Bellovin, Josh Benaloh, Matt Blaze, Jon Callas, Whitfield Diffie, Susan Landau, Peter G. Neumann, Ronald L. Rivest, Jeffrey I. Schiller, Bruce Schneier, Vanessa Teague, and Carmela Troncoso. Bugs in our pockets: The risks of client-side scanning. CoRR , abs/2110.07450, 2021. - · arXiv
  2. [13][13] Noam Eshed. Novelty detection and analysis in convolutional neural networks. Master's thesis, Cornell University, 2020. -
  3. [19][19] Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. In NeurIPS , volume 27, 2014. -
  4. [20][20] Ian Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. In ICLR , 2015. -
  5. [31][31] Aditya Khosla, Nityananda Jayadevaprakash, Bangpeng Yao, and Li Fei-Fei. Novel dataset for fine-grained image categorization. In First Workshop on Fine-Grained Visual Categorization, CVPR , 2011. -
  6. [39][39] Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. In ICLR , 2018. -
  7. [41][41] Official Journal of the European Union. Regulation (EU) 2021/1232, 2021. https://eur-lex.europa. eu/legal-content/EN/TXT/?uri=CELEX: 32021R1232 , accessed: Nov. 16, 2021. -
  8. [43][43] Benny Pinkas. A review of the cryptography behind the apple psi system, 2021. https://www.apple.com/ child-safety/pdf/Technical\_Assessment\_ of\_CSAM\_Detection\_Benny\_Pinkas.pdf , accessed: Oct. 1, 2021. -
  9. [46][46] Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. Intriguing properties of neural networks. In ICLR , 2014. -
  10. [56][56] F. Zhao, Yongzhen Huang, Liang Wang, and Tieniu Tan. Deep semantic ranking based hashing for multi-label image retrieval. In CVPR , pages 1556-1564, 2015.

Cited by

Sorted by citations

Authors
Struppek, L., Hintersdorf, D., Neider, D., & Kersting, K.
Affiliations
  • Technische Universität Darmstadt (DE)
  • Carl von Ossietzky Universität Oldenburg (DE)
  • Department of Computer Science and Centre for Cognitive Science, Technical University of Darmstadt, Germany and Hessian Center for Artificial Intelligence, Germany
Venue
2022 ACM Conference on Fairness Accountability and Transparency
Publisher
ACM
Published
2022-06-20
Pages
58-69
OpenAlex
W3214430180
Cited by
153
References
36
Citations by year
2026: 12 · 2025: 11 · 2024: 14 · 2023: 112 · 2022: 4
FWCI
12.78
Type
conference paper
Language
en
Open access
Locations
  • ACM Conference on Fairness, Accountability, and Transparency (FAccT) (publishedVersion, cc-by-sa)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Funders
  • Bundesministerium für Bildung und Forschung (13N15343)
  • National Research Center for Applied Cybersecurity ATHENE
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Digital Media Forensic Detection
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Steganography and Watermarking Techniques
  • Physical Sciences › Computer Science › Artificial Intelligence › Privacy-Preserving Technologies in Data
Keywords
hash collision attacks, adversarial attacks, privacy, iCloud, image editing, hash robustness

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review