CoChat
Sign Up
PreprintCited by 15Advanced Steganography and Watermarking TechniquesDigital Media Forensic DetectionGenerative Adversarial Networks and Image Synthesis

The DOI matches its DataCite record.

DiffusionShield: A Watermark for Copyright Protection against Generative Diffusion Models

Open access
Cui, Yingqian, Ren, Jie, Xu, Han, He, Pengfei, Liu, Hui, Sun, Lichao, Xing, Yue, Tang, Jiliang·2023·arXiv
DOI 10.48550/arxiv.2306.04642arXiv 2306.04642
Open this paper in CoChat — free accountOpen PDF

Abstract

Recently, Generative Diffusion Models (GDMs) have showcased their remarkable capabilities in learning and generating images. A large community of GDMs has naturally emerged, further promoting the diversified applications of GDMs in various fields. However, this unrestricted proliferation has raised serious concerns about copyright protection. For example, artists including painters and photographers are becoming increasingly concerned that GDMs could effortlessly replicate their unique creative works without authorization. In response to these challenges, we introduce a novel watermarking scheme, DiffusionShield, tailored for GDMs. DiffusionShield protects images from copyright infringement by GDMs through encoding the ownership information into an imperceptible watermark and injecting it into the images. Its watermark can be easily learned by GDMs and will be reproduced in their generated images. By detecting the watermark from generated images, copyright infringement can be exposed with evidence. Benefiting from the uniformity of the watermarks and the joint optimization method, DiffusionShield ensures low distortion of the original image, high watermark detection performance, and the ability to embed lengthy messages. We conduct rigorous and comprehensive experiments to show the effectiveness of DiffusionShield in defending against infringement by GDMs and its superiority over traditional watermarking methods. The code for DiffusionShield is accessible in https://github.com/Yingqiancui/DiffusionShield.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2306.04642

Cites

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Detection efforts have expanded from passive detection to proactive solutions, which disrupt unauthorized deepfake generation via adversarial perturbations or watermarking.

    “Additionally, detection efforts have expanded from passive detection to proactive solutions [14], [15], which disrupt unauthorized deepfake generation via adversarial perturbations or watermarking.”

    From Evolving from Single-modal to Multi-modal Facial Deepfake Detection: Progress and Challenges (2024) · cites this paper as [15]

    The Abstract (p. 1) describes a proactive watermark placed in images before diffusion-model training, but says its purpose is to expose infringement by detecting that watermark in generated images. It does not say DiffusionShield disrupts unauthorized deepfake generation through adversarial perturbations; its focus is copyright protection against generative diffusion models. (LLM, conf=0.87)

    Experimental — accuracy not yet measured

  • Early diffusion-specific defenses emphasized robust watermarking within the latent diffusion process, embedding uniform and traceable signals directly into latent spaces or diffusion steps.

    “Early diffusionspecific defenses emphasized robust watermarking within the latent diffusion process, embedding uniform and traceable signals directly into latent spaces or diffusion steps [15], [213], [214], [215].”

    From Evolving from Single-modal to Multi-modal Facial Deepfake Detection: Progress and Challenges (2024) · cites this paper as [15]

    The Introduction (p. 2) and §3.3 (p. 4) describe uniform, owner-specific watermarks that encode copyright messages and can be detected in diffusion-model outputs. But the watermark is applied to owners’ images, not embedded directly into latent spaces or diffusion steps as the citing sentence suggests. (LLM, conf=0.88)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

No references could be resolved yet.

Research works printed but not linked: 40

  1. Allen-Zhu, Z. and Li, Y. Feature purification: How adversarial training performs robust deep learning. In 2021 IEEE 62nd Annual Symposium on Foundations of Computer Science (FOCS) , pp. 977-988. IEEE, 2022.
  2. Arora, S., Li, Y ., Liang, Y ., Ma, T., and Risteski, A. A latent variable model approach to pmi-based word embeddings. Transactions of the Association for Computational Linguistics , 4:385-399, 2016.
  3. Ba, J., Erdogdu, M., Suzuki, T., Wu, D., and Zhang, T. Generalization of two-layer neural networks: An asymptotic viewpoint. In International conference on learning representations , 2019.
  4. Chen, T., Kornblith, S., Norouzi, M., and Hinton, G. A simple framework for contrastive learning of visual representations. In International conference on machine learning , pp. 1597-1607. PMLR, 2020.
  5. Cox, I., Miller, M., Bloom, J., and Honsinger, C. Digital watermarking. Journal of Electronic Imaging , 11(3):414414, 2002.
  6. Daras, G., Dagan, Y., Dimakis, A. G., and Daskalakis, C. Consistent diffusion models: Mitigating sampling drift by learning to be consistent. arXiv preprint arXiv:2302.09057 , 2023. · arXiv
  7. Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., and Fei-Fei, L. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition , pp. 248-255. Ieee, 2009.
  8. Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., and Bengio, Y. Generative adversarial networks. Communications of the ACM , 63(11):139-144, 2020.
  9. Guo, Y., Zhang, L., Hu, Y., He, X., and Gao, J. Ms-celeb-1m: A dataset and benchmark for large-scale face recognition. In European conference on computer vision , pp. 87-102. Springer, 2016.
  10. He, K., Zhang, X., Ren, S., and Sun, J. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition , pp. 770-778, 2016.
  11. Ho, J. and Salimans, T. Classifier-free diffusion guidance. arXiv preprint arXiv:2207.12598 , 2022. · arXiv
  12. Ho, J., Jain, A., and Abbeel, P. Denoising diffusion probabilistic models. Advances in Neural Information Processing Systems , 33:6840-6851, 2020.
  13. Jia, Z., Fang, H., and Zhang, W. Mbrs: Enhancing robustness of dnn-based watermarking by mini-batch of real and simulated jpeg compression. In Proceedings of the 29th ACM international conference on multimedia , pp. 41-49, 2021.
  14. Kumar, A. A review on implementation of digital image watermarking techniques using lsb and dwt. Information and Communication Technology for Sustainable Development: Proceedings of ICT4SD 2018 , pp. 595-602, 2020.
  15. Ma, R., Guo, M., Hou, Y., Yang, F., Li, Y., Jia, H., and Xie, X. Towards blind watermarking: Combining invertible and non-invertible mechanisms. In Proceedings of the 30th ACM International Conference on Multimedia , pp. 1532-1542, 2022.
  16. Madry, A., Makelov, A., Schmidt, L., Tsipras, D., and Vladu, A. Towards deep learning models resistant to adversarial attacks. arXiv preprint arXiv:1706.06083 , 2017. · arXiv
  17. Mairal, J., Bach, F., Ponce, J., and Sapiro, G. Online learning for matrix factorization and sparse coding. Journal of Machine Learning Research , 11(1), 2010.
  18. Navas, K., Ajay, M. C., Lekshmi, M., Archana, T. S., and Sasikumar, M. Dwt-dct-svd based watermarking. In 2008 3rd International Conference on Communication Systems Software and Middleware and Workshops (COMSWARE'08) , pp. 271-274. IEEE, 2008.
  19. Nichol, A. Q. and Dhariwal, P. Improved denoising diffusion probabilistic models. In International Conference on Machine Learning , pp. 8162-8171. PMLR, 2021.
  20. Olshausen, B. A. and Field, D. J. Sparse coding with an overcomplete basis set: A strategy employed by v1? Vision research , 37(23):3311-3325, 1997.
  21. Podilchuk, C. I. and Delp, E. J. Digital watermarking: algorithms and applications. IEEE signal processing Magazine , 18(4):33-46, 2001.
  22. Ramesh, A., Dhariwal, P., Nichol, A., Chu, C., and Chen, M. Hierarchical text-conditional image generation with clip latents. arXiv preprint arXiv:2204.06125 , 2022. · arXiv
  23. Rombach, R., Blattmann, A., Lorenz, D., Esser, P., and Ommer, B. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pp. 10684-10695, 2022.
  24. Sehwag, V., Hazirbas, C., Gordo, A., Ozgenel, F., and Canton, C. Generating high fidelity data from low-density regions using diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pp. 11492-11501, 2022.
  25. Shih, F. Y. and Wu, S. Y. Combinational image watermarking in the spatial and frequency domains. Pattern Recognition , 36(4):969-975, 2003.
  26. Song, J., Meng, C., and Ermon, S. Denoising diffusion implicit models. arXiv preprint arXiv:2010.02502 , 2020. · arXiv
  27. Tancik, M., Mildenhall, B., and Ng, R. Stegastamp: Invisible hyperlinks in physical photographs. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pp. 2117-2126, 2020.
  28. Um, S. and Ye, J. C. Don't play favorites: Minority guidance for diffusion models. arXiv preprint arXiv:2301.12334 , 2023. · arXiv
  29. Weng, X., Li, Y., Chi, L., and Mu, Y. High-capacity convolutional video steganography with temporal residual modeling. In Proceedings of the 2019 on international conference on multimedia retrieval , pp. 87-95, 2019.
  30. Wu, H., Liu, G., Yao, Y., and Zhang, X. Watermarking neural networks with watermarked images. IEEE Transactions on Circuits and Systems for Video Technology , 31 (7):2591-2601, 2020.
  31. Yu, F., Seff, A., Zhang, Y., Song, S., Funkhouser, T., and Xiao, J. Lsun: Construction of a large-scale image dataset using deep learning with humans in the loop. arXiv preprint arXiv:1506.03365 , 2015. · arXiv
  32. Yu, N., Skripniuk, V., Abdelnabi, S., and Fritz, M. Artificial fingerprinting for generative models: Rooting deepfake attribution in training data. In Proceedings of the IEEE/CVF International conference on computer vision , pp. 14448-14457, 2021.
  33. Zhang, H., Wang, H., Cao, Y., Shen, C., and Li, Y. Robust data hiding using inverse gradient attention. arXiv preprint arXiv:2011.10850 , 2020a. · arXiv
  34. Zhang, J., Chen, D., Liao, J., Fang, H., Zhang, W., Zhou, W., Cui, H., and Yu, N. Model watermarking for image processing networks. In Proceedings of the AAAI conference on artificial intelligence , volume 34, pp. 12805-12812, 2020b.
  35. Zhang, L. and Agrawala, M. Adding conditional control to text-to-image diffusion models. arXiv preprint arXiv:2302.05543 , 2023. · arXiv
  36. Zhang, R., Isola, P., Efros, A. A., Shechtman, E., and Wang, O. The unreasonable effectiveness of deep features as a perceptual metric. In Proceedings of the IEEE conference on computer vision and pattern recognition , pp. 586-595, 2018.
  37. Zhao, X., Zhang, K., Wang, Y.-X., and Li, L. Generative autoencoders as watermark attackers: Analyses of vulnerabilities and threats. arXiv preprint arXiv:2306.01953 , 2023a. · arXiv
  38. Zhao, Y., Liu, B., Ding, M., Liu, B., Zhu, T., and Yu, X. Proactive deepfake defence via identity watermarking. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision , pp. 4602-4611, 2023b.
  39. Zhao, Y., Pang, T., Du, C., Yang, X., Cheung, N.-M., and Lin, M. A recipe for watermarking diffusion models. arXiv preprint arXiv:2303.10137 , 2023c. · arXiv
  40. Zhu, J., Kaplan, R., Johnson, J., and Fei-Fei, L. Hidden: Hiding data with deep networks. In Proceedings of the European conference on computer vision (ECCV) , pp. 657-672, 2018.

Cited by

Sorted by citations

Metadata

Authors
Cui, Y., Ren, J., Xu, H., He, P., Liu, H., Sun, L., Xing, Y., & Tang, J.
ORCID
Venue
arXiv
Publisher
arXiv
Published
2023
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.2306.04642
arXiv
2306.04642
OpenAlex
W4380135882
Cited by
15
References
0
Citations by year
2025: 6 · 2024: 9
Type
posted content
Language
en
Open access
Licence
Locations
  • arXiv (Cornell University) (submittedVersion, cc-by)
  • arXiv (Cornell University) (cc-by)
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Steganography and Watermarking Techniques
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Digital Media Forensic Detection
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Generative Adversarial Networks and Image Synthesis
Keywords
image watermarking, copyright protection, generative diffusion models, watermark detection, watermark embedding, copyright infringement, joint optimization

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review