Cited by 40Advanced Steganography and Watermarking TechniquesSpeech Recognition and SynthesisSpeech and Audio Processing

Detecting Voice Cloning Attacks via Timbre Watermarking

Open access
Liu, Chang, Zhang, Jie, Zhang, Tianwei, Yang, Xi, Zhang, Weiming, Yu, NengHai·2024·Proceedings 2024 Network and Distributed System Security Symposium

Abstract

Nowadays, it is common to release audio content to the public, for social sharing or commercial purposes.However, with the rise of voice cloning technology, attackers have the potential to easily impersonate a specific person by utilizing his publicly released audio without any permission.Therefore, it becomes significant to detect any potential misuse of the released audio content and protect its timbre from being impersonated.To this end, we introduce a novel concept, "Timbre Watermarking", which embeds watermark information into the target individual's speech, eventually defeating the voice cloning attacks.However, there are two challenges: 1) robustness: the attacker can remove the watermark with common speech preprocessing before launching voice cloning attacks; 2) generalization: there are a variety of voice cloning approaches for the attacker to choose, making it hard to build a general defense against all of them.To address these challenges, we design an end-to-end voice cloning-resistant detection framework.The core idea of our solution is to embed the watermark into the frequency domain, which is inherently robust against common data processing methods.A repeated embedding strategy is adopted to further enhance the robustness.To acquire generalization across different voice cloning attacks, we modulate their shared process and integrate it into our framework as a distortion layer.Experiments demonstrate that the proposed timbre watermarking can defend against different voice cloning attacks, exhibit strong resistance against various adaptive attacks (e.g., reconstruction-based removal attacks, watermark overwriting attacks), and achieve practicality in real-world services such as PaddleSpeech, Voice-Cloning-App, and so-vits-svc.In addition, ablation studies are also conducted to verify the effectiveness of our design.Some audio samples are available at https://timbrewatermarking.github.io/samples.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.14722/ndss.2024.24200

Cites

64 of 64 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • The method exhibits strong resistance to advanced adversarial attacks targeting watermark integrity, such as watermark overwriting and removal attacks.

    “Furthermore, the method exhibits strong resistance to advanced adversarial attacks targeting watermark integrity, such as watermark overwriting and removal attacks.”

    From Audio Deepfake Detection: What Has Been Achieved and What Lies Ahead (2025) · cites this paper as [171]

    The Watermark Overwriting Attack section reports that the default method fails against an insider overwriting attack, although an enhanced strategy achieves 100% extraction accuracy. The Audio Reconstruction-based Removal Attacks section reports that a removal attempt left 74.90% extraction accuracy and produced poor-quality speech. The claim captures resistance to removal and to overwriting with the enhanced strategy, but omits the default method’s vulnerability. (LLM, conf=0.84)

    Experimental — accuracy not yet measured
  • Watermark extraction accuracy remains at 100% against preprocessing attacks such as resampling and compression while maintaining nearly 90% accuracy under extreme conditions like low-bitrate compression and low-pass filtering.

    “Experimental results show that the watermark extraction accuracy remains at 100% against preprocessing attacks such as resampling and compression while maintaining nearly 90% accuracy under extreme conditions like low-bitrate compression and low-pass filtering.”
  • Even when only 75% of the training data are watermarked, cloned voices can still be effectively detected.

    “Even when only 75% of the training data are watermarked, cloned voices can still be effectively detected.”

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 7

  1. [10]10. M. E. Ahmed, I.-Y. Kwak, J. H. Huh, I. Kim, T. Oh, and H. Kim, "Void: A fast and light voice liveness detection system," in Proceedings of the 29th USENIX Conference on Security Symposium, 2020, pp. 2685- 2702.
  2. [28]28. J. Kim, J. Kong, and J. Son, "Conditional variational autoencoder with adversarial learning for end-to-end text-to-speech," in International Conference on Machine Learning. PMLR, 2021, pp. 5530-5540.
  3. [33]33. W. Ping, K. Peng, A. Gibiansky, S. O. Arik, A. Kannan, S. Narang, J. Raiman, and J. Miller, "Deep voice 3: 2000-speaker neural text-to- speech," proc. ICLR, pp. 214-217, 2018.
  4. [54]54. I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio, "Generative adversarial net- works," Communications of the ACM, vol. 63, no. 11, pp. 139-144, 2020.
  5. [68]68. C. D. Kim, B. Kim, H. Lee, and G. Kim, "Audiocaps: Generating captions for audios in the wild," in Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2019, pp. 119-132.
  6. [69]69. H. Zhang, T. Yuan, J. Chen, X. Li, R. Zheng, Y. Huang, X. Chen, E. Gong, Z. Chen, X. Hu et al., "Paddlespeech: An easy-to-use all-in- one speech toolkit," in Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies: System Demonstrations, 2022, pp. 114-123.
  7. [76]76. Y. Chen, X. Yuan, J. Zhang, Y. Zhao, S. Zhang, K. Chen, and X. Wang, "Devil's whisper: A general approach for physical adversarial attacks against commercial black-box speech recognition devices." in USENIX Security Symposium, 2020, pp. 2667-2684.

Cited by

Sorted by citations

Authors
Liu, C., Zhang, J., Zhang, T., Yang, X., Zhang, W., & Yu, N.
Affiliations
  • University of Science and Technology of China (CN)
  • Nanyang Technological University (SG)
Venue
Proceedings 2024 Network and Distributed System Security Symposium
Published
2024
OpenAlex
W4391725339
Cited by
40
References
63
Citations by year
2026: 19 · 2025: 13 · 2024: 8
FWCI
8.23
Type
conference paper
Language
en
Open access
Licence
public-domain
Locations
  • Network and Distributed System Security Symposium (publishedVersion, public-domain)
  • arXiv (Cornell University) (submittedVersion, public-domain)
  • arXiv (Cornell University)
Funders
  • National Natural Science Foundation of China (U20B2047, 62121002, 62002334, 62072421, 62102386)
  • Ministry of Education, India
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Steganography and Watermarking Techniques
  • Physical Sciences › Computer Science › Artificial Intelligence › Speech Recognition and Synthesis
  • Physical Sciences › Computer Science › Signal Processing › Speech and Audio Processing
Keywords
voice spoofing detection, watermark robustness, frequency domain embedding, adaptive attacks, watermark removal attacks

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review