D-CAPTCHA++: A Study of Resilience of Deepfake CAPTCHA under Transferable Imperceptible Adversarial Attack
Open accessAbstract
The advancements in generative AI have enabled the improvement of audio synthesis models, including text-to-speech and voice conversion. This raises concerns about its potential misuse in social manipulation and political interference, as synthetic speech has become indistinguishable from natural human speech. Several speech-generation programs are utilized for malicious purposes, especially impersonating individuals through phone calls. Therefore, detecting fake audio is crucial to maintain social security and safeguard the integrity of information. Recent research has proposed a D-CAPTCHA system based on the challenge-response protocol to differentiate fake phone calls from real ones. In this work, we study the resilience of this system and introduce a more robust version, D-CAPTCHA++, to defend against fake calls. Specifically, we first expose the vulnerability of the D-CAPTCHA system under the transferable imperceptible adversarial attack. Secondly, we mitigate such vulnerability by improving the robustness of the system by using adversarial training in D-CAPTCHA deepfake detectors and task classifiers.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Adversarial training has been demonstrated to be effective across both visual and audio modalities.
“This technique has been demonstrated to be effective across both visual and audio modalities [202, 203].”
From Deepfake detection across image, video, and audio: a comprehensive survey with empirical evaluation of generalization and robustness (2026) · cites this paper as [203]
Section 4.3.3 reports that PGD adversarial training substantially reduced attack success against the paper’s audio deepfake detectors and task classifiers. The paper does not demonstrate its effectiveness in visual applications, so it supports the audio portion but not the claim about both modalities. (LLM, conf=0.88)
Experimental — accuracy not yet measuredPure adversarial training cuts WA ASR from 81.06% to 37.03% and TA ASR from 34.17% to 2.01%.
“Pure adversarial training [202, 203] cuts WA ASR from 81 . 06% to 37 . 03% and TA ASR from 34 . 17% to 2 . 01%”
From Deepfake detection across image, video, and audio: a comprehensive survey with empirical evaluation of generalization and robustness (2026) · cites this paper as [203]
Section 4.3.3 reports that PGD adversarial training substantially reduced attack success rates, supporting the general direction of the claim. It reports different figures—32.26% to 2.27% for deepfake detectors and 31.31% to 0.60% for task classifiers—and does not identify the claimed WA and TA figures. No page number is available in our copy. (LLM, conf=0.86)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · see citing works
Towards Deep Learning Models Resistant to Adversarial Attacks
Madry, A., Makelov, A., Schmidt, L., Tsipras, D., & Vladu, A. (2017) · arXiv · doi:10.48550/arxiv.1706.06083 · Cited by 6,602 · see citing works
Audio Set: An ontology and human-labeled dataset for audio events
Gemmeke, J. F., Ellis, D. P. W., Freedman, D., Jansen, A., Lawrence, W., Moore, R. C., Plakal, M., & Ritter, M. (2017) · 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2017.7952261 · Cited by 3,134 · see citing works
Livingstone, S. R., & Russo, F. (2018) · PLoS ONE · doi:10.1371/journal.pone.0196391 · Cited by 1,955 · see citing works
Transferability in Machine Learning: from Phenomena to Black-Box Attacks using Adversarial Samples
Papernot, N., McDaniel, P., & Goodfellow, I. (2016) · arXiv · doi:10.48550/arxiv.1605.07277 · Cited by 1,326 · see citing works
CREMA-D: Crowd-Sourced Emotional Multimodal Actors Dataset
Cao, H., Cooper, D. G., Keutmann, M. K., Gur, R. C., Nenkova, A., & Verma, R. (2014) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2014.2336244 · Cited by 954 · see citing works
ASVspoof 2019: Future Horizons in Spoofed and Fake Audio Detection
Todisco, M., Wang, X., Vestman, V., Sahidullah, M., Delgado, H., Nautsch, A., Yamagishi, J., Evans, N., Kinnunen, T. H., & Lee, K. A. (2019) · Interspeech 2019 · doi:10.21437/interspeech.2019-2249 · Cited by 687 · see citing works
End-to-End anti-spoofing with RawNet2
Tak, H., Patino, J., Todisco, M., Nautsch, A., Evans, N., & Larcher, A. (2021) · ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp39728.2021.9414234 · Cited by 402 · see citing works
ASVspoof 2021: accelerating progress in spoofed and deepfake speech detection
Yamagishi, J., Wang, X., Todisco, M., Sahidullah, M., Patino, J., Nautsch, A., Liu, X., Lee, K. A., Kinnunen, T., Evans, N., et al. (2021) · 2021 Edition of the Automatic Speaker Verification and Spoofing Countermeasures Challenge · doi:10.21437/asvspoof.2021-8 · Cited by 380 · see citing works
CSTR VCTK Corpus: English Multi-speaker Corpus for CSTR Voice Cloning Toolkit (version 0.92)
Yamagishi, J., Veaux, C., & MacDonald, K. (2019) · University of Edinburgh. The Centre for Speech Technology Research (CSTR) · doi:10.7488/ds/2645 · Cited by 356 · see citing works
Imperceptible, Robust, and Targeted Adversarial Examples for Automatic Speech RecognitionPreprint
Qin, Y., Carlini, N., Goodfellow, I., Cottrell, G. W., & Raffel, C. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1903.10346 · Cited by 292 · see citing works
Why Do Adversarial Attacks Transfer? Explaining Transferability of Evasion and Poisoning AttacksPreprint
Demontis, A., Melis, M., Pintor, M., Jagielski, M., Biggio, B., Oprea, A., Nita-Rotaru, C., & Roli, F. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1809.02861 · Cited by 219 · see citing works
Targeted Adversarial Examples for Black Box Audio Systems
Taori, R., Kamsetty, A., Chu, B., & Vemuri, N. (2019) · IEEE Security and Privacy Workshops (SPW) · doi:10.1109/spw.2019.00016 · Cited by 190 · see citing works
The GTZAN dataset: Its contents, its faults, their effects on evaluation, and its future usePreprint
Sturm, B. L. T. (2013) · VBN Forskningsportal (Aalborg Universitet) · doi:10.48550/arxiv.1306.1461 · Cited by 156 · see citing works
Wu, Z., Das, R. K., Yang, J., & Li, H. (2020) · Interspeech (USB) · doi:10.21437/interspeech.2020-1810 · Cited by 115 · see citing works
Pushing the limits of raw waveform speaker recognition
Jung, J.-W., Kim, Y., Heo, H.-S., Lee, B., Kwon, Y., & Chung, J. S. (2022) · Interspeech (USB) · doi:10.21437/interspeech.2022-126 · Cited by 100 · see citing works
Sound Event Detection in Synthetic Domestic Environments
Serizel, R., Turpault, N., Shah, A., & Salamon, J. (2020) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp40776.2020.9054478 · Cited by 99 · see citing works
Voice Conversion With Just Nearest Neighbors
Baas, M., van Niekerk, B., & Kamper, H. (2023) · Interspeech (USB) · doi:10.21437/interspeech.2023-419 · Cited by 77 · see citing works
WaveFake: A Data Set to Facilitate Audio Deepfake Detection
Frank, J., & Schönherr, L. (2021) · arXiv · doi:10.48550/arxiv.2111.02813 · Cited by 76 · see citing works
Towards Query-Efficient Adversarial Attacks Against Automatic Speech Recognition Systems
Wang, Q., Zheng, B., Li, Q. L., Shen, C. Q., & Ba, Z. (2020) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2020.3026543 · Cited by 62 · see citing works
Vocalset: A Singing Voice Dataset
Wilkins, J., Seetharaman, P., Wahl, A., & Pardo, B. (2018) · Figshare · doi:10.5281/zenodo.1193956 · Cited by 60 · see citing works
Vocalsound: A Dataset for Improving Human Vocal Sounds Recognition
Gong, Y., Yu, J., & Glass, J. (2022) · ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp43922.2022.9746828 · Cited by 43 · cited 1× · see citing works
“In our case, we use the LA subset, which consists of 12 , 483 bonafide and 108 , 978 fake audio samples.”
MethodSpecRNet: Towards Faster and More Accessible Audio DeepFake Detection
Kawa, P., Plata, M., & Syga, P. (2022) · IEEE International Conference on Trust, Security and Privacy in Computing and Communications (TrustCom) · doi:10.1109/trustcom56396.2022.00111 · Cited by 36 · see citing works
Lin, Y., & Abdulla, W. H. (2014) · Springer eBooks · doi:10.1007/978-3-319-07974-5_2 · Cited by 35 · see citing works
TriAAN-VC: Triple Adaptive Attention Normalization for Any-to-Any Voice Conversion
Park, H. J., Yang, S. W., Kim, J. S., Shin, W., & Han, S. W. (2023) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49357.2023.10096642 · Cited by 24 · see citing works
Defense Against Adversarial Attacks on Audio DeepFake Detection
Kawa, P., Plata, M., & Syga, P. (2023) · INTERSPEECH 2023 · doi:10.21437/interspeech.2023-409 · Cited by 19 · see citing works
Deepfake CAPTCHA: A Method for Preventing Fake Calls
Yasur, L., Frankovits, G., Grabovski, F. M., & Mirsky, Y. (2023) · ACM Asia Conference on Computer and Communications Security (ASIA CCS) · doi:10.1145/3579856.3595801 · Cited by 18 · see citing works
Humtrans: A Novel Open-Source Dataset for Humming Melody Transcription and Beyond
Liu, S., Xu, L., Li, D., & Shan, Y. (2024) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp48485.2024.10445818 · Cited by 8 · see citing works
Rhythm Modeling for Voice Conversion
van Niekerk, B., Carbonneau, M., & Kamper, H. (2023) · IEEE Signal Processing Letters · doi:10.1109/lsp.2023.3313515 · Cited by 4 · see citing works
Flagged first, then by citations
Research works printed but not linked: 3
- W. X. Zhao, K. Zhou, J. Li, T. Tang, X. Wang, Y. Hou, Y. Min, B. Zhang, J. Zhang, Z. Dong et al., "A survey of large language models," arXiv preprint arXiv:2303.18223, 2023. · arXiv
- B. Sisman, J. Yamagishi, S. King, and H. Li, "An overview of voice conversion and its challenges: From statistical modeling to deep learning," IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 29, pp. 132-157, 2020.
- T.-Y. Lin, P. Goyal, R. Girshick, K. He, and P. Dollár, "Focal loss for dense object detection," in Proceedings of the IEEE international conference on computer vision, 2017, pp. 2980-2988.
Cited by
Sorted by citations
PITCH: AI-assisted Tagging of Deepfake Audio Calls using Challenge-Response
Mittal, G., Jakobsson, A., Marshall, K. A., Hegde, C., & Memon, N. D. (2025) · ACM Asia Conference on Computer and Communications Security (ASIA CCS) · doi:10.1145/3708821.3733861 · Cited by 12 · see citing works
Nguyen-Le, H.-H., Tran, V.-T., Nguyen, D.-T., & Le-Khac, N.-A. (2026) · Artificial Intelligence Review · doi:10.1007/s10462-026-11608-4 · Cited by 1 · see citing works
Active Acoustic Probes for AI Voice Call Detection
Yasur, L., & Mirsky, Y. (2026) · ACM Workshop on Deepfake, Deception, and Disinformation Security (3D-Sec) · doi:10.1145/3843205.3845633
An Intelligent Captcha Interpretation and Speech Based Human Verification Model
Kulkarni, H. N., M, D. K., Nagashree, K., Manisha, M., & Podaralla, R. (2026) · doi:10.1109/nmitcon70171.2026.11754472
- Authors
- Nguyen-Le, H.-H., Tran, V.-T., Nguyen, D.-T., & Le-Khac, N.-A.
- Affiliations
- University College Dublin (IE)
- Trinity College Dublin (IE)
- University of Science,Vietnam
- University College Dublin,Ireland
- ORCID
- Nguyen-Le, Hong-Hanh
- Venue
- 2024 International Joint Conference on Neural Networks (IJCNN)
- Publisher
- IEEE
- Published
- 2024-06-30
- Pages
- 1-8
- ISSN
- 2161-4407 (ISSN-L)
- OpenAlex
- W4402351610
- Cited by
- 2
- References
- 29
- Citations by year
- 2026: 1 · 2025: 1
- FWCI
- 0.59
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- International Joint Conference on Neural Networks (IJCNN) (publishedVersion)
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Funders
- Science Foundation Ireland
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
- Physical Sciences › Computer Science › Artificial Intelligence › Speech Recognition and Synthesis
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Digital Media Forensic Detection
- Keywords
- deepfake detection, adversarial attacks, transferable adversarial attacks, adversarial training, robustness, speech synthesis, text-to-speech, voice conversion, challenge-response protocol, social manipulation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
