Complex-valued neural networks for voice anti-spoofing
Open accessThe DOI matches its Crossref record; 31 of the 31 references it cites were found in a registry or an index.
Abstract
Current anti-spoofing and audio deepfake detection systems use either magnitude spectrogram-based features (such as CQT or Melspectrograms) or raw audio processed through convolution or sinc-layers. Both methods have drawbacks: magnitude spectrograms discard phase information, which affects audio naturalness, and raw-feature-based models cannot use traditional explainable AI methods. This paper proposes a new approach that combines the benefits of both methods by using complex-valued neural networks to process the complex-valued, CQT frequency-domain representation of the input audio. This method retains phase information and allows for explainable AI methods. Results show that this approach outperforms previous methods on the "In-the-Wild" anti-spoofing dataset and enables interpretation of the results through explainable AI. Ablation studies confirm that the model has learned to use phase information to detect voice spoofing.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Müller et al. [107] proposed Complex-Valued CQT Spectrograms (C-CQTs), which retain both magnitude and phase information. Using a complex-valued neural network (CVNN) classifier based on CNNs, this approach achieved SOTA performance on the ASVspoof2019 LA and IWA datasets”
From Audio Deepfake Detection: What Has Been Achieved and What Lies Ahead (2025) · cites this paper as [107]
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404 · cited 2×
“This dataset is widely used in related research [4, 5, 6, 9, 10, 11, 12, 13] and is considered one of the most established anti-spoofing datasets.”
MethodVery Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148 · cited 2×
“This is because existing XAI techniques such as saliency maps [16] or Smooth Grad [17] are designed for spatially-dimensioned input, while “raw” audio is a one-dimensional vector.”
MethodExploiting Generative AI to Scale up Intelligent Tutoring Systems
Jakubův, J., Chvalovský, K., Goertzel, Z., Kaliszyk, C., Olšák, M., Piotrowski, B., Schulz, S., Suda, M., & Urban, J. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.itp.2023.19 · Cited by 76,658
Explaining and Harnessing Adversarial Examples
Goodfellow, I. J., Shlens, J., & Szegedy, C. (2014) · arXiv · doi:10.48550/arxiv.1412.6572 · Cited by 10,381 · cited 2×
“First, we use Adversarial Retraining [31].”
MethodOn the Properties of Neural Machine Translation: Encoder–Decoder Approaches
Cho, K., van Merrienboer, B., Bahdanau, D., & Bengio, Y. (2014) · Proceedings of SSST-8, Eighth Workshop on Syntax, Semantics and Structure in Statistical Translation · doi:10.3115/v1/w14-4012 · Cited by 7,099 · cited 1×
“However, more complex components such as attention and recurrence, such as GRU [25], need to be adapted, since typical activation functions like sigmoid , tanh , and softmax cannot be applied directly to complex-valued scalars [26].”
BackgroundWaveNet: a generative model for raw audio
Oord, A. V. D., Dieleman, S., Zen, H., Simonyan, K., Vinyals, O., Graves, A., Kalchbrenner, N., Senior, A., & Kavukcuoglu, K. (2016) · arXiv · doi:10.48550/arxiv.1609.03499 · Cited by 4,911 · cited 1×
“This phase information is crucial for audio quality and voice naturalness [1], and in the domain of speech-to-text (STT), it is laboriously recreated us-ing Griffin-Lim or neural vocoders [1, 2, 3].”
MethodShortcut learning in deep neural networks
Geirhos, R., Jacobsen, J.-H., Michaelis, C., Zemel, R., Brendel, W., Bethge, M., & Wichmann, F. A. (2020) · Nature Machine Intelligence · doi:10.1038/s42256-020-00257-z · Cited by 2,224 · cited 1×
“Among other disadvantages, this impedes the detection of “learning shortcuts” [18], i.e. highly predictive, but artificial correlations between input and target which do not reflect real-world causality.”
BackgroundSignal estimation from modified short-time Fourier transform
Griffin, D., & Lim, J. (1984) · IEEE Transactions on Acoustics, Speech, and Signal Processing · doi:10.1109/tassp.1984.1164317 · Cited by 2,095
MesoNet: a Compact Facial Video Forgery Detection Network
Afchar, D., Nozick, V., Yamagishi, J., & Echizen, I. (2018) · 2018 IEEE International Workshop on Information Forensics and Security (WIFS) · doi:10.1109/wifs.2018.8630761 · Cited by 1,872
Introduction to the Psychology of Hearing
OWENS, E. (1977) · Archives of Otolaryngology - Head and Neck Surgery · doi:10.1001/archotol.1977.00780290081022 · Cited by 1,766
SmoothGrad: removing noise by adding noise
Smilkov, D., Thorat, N., Kim, B., Viégas, F., & Wattenberg, M. (2017) · arXiv · doi:10.48550/arxiv.1706.03825 · Cited by 1,467 · cited 2×
“This is because existing XAI techniques such as saliency maps [16] or Smooth Grad [17] are designed for spatially-dimensioned input, while “raw” audio is a one-dimensional vector.”
MethodSpeaker Recognition from Raw Waveform with SincNet
Ravanelli, M., & Bengio, Y. (2018) · 2018 IEEE Spoken Language Technology Workshop (SLT) · doi:10.1109/slt.2018.8639585 · Cited by 896
ASVspoof 2019: Future Horizons in Spoofed and Fake Audio Detection
Todisco, M., Wang, X., Vestman, V., Sahidullah, M., Delgado, H., Nautsch, A., Yamagishi, J., Evans, N., Kinnunen, T., & Lee, K. A. (2019) · arXiv · doi:10.48550/arxiv.1904.05441 · Cited by 688 · cited 2×
“We adopt the Equal Error Rate (EER) as the performance metric for our model evaluation, consistent with previous studies [4, 5, 6, 28, 36].”
BackgroundResultASVspoof 2019: Future Horizons in Spoofed and Fake Audio Detection
Todisco, M., Wang, X., Vestman, V., Sahidullah, M., Delgado, H., Nautsch, A., Yamagishi, J., Evans, N., Kinnunen, T. H., & Lee, K. A. (2019) · Interspeech 2019 · doi:10.21437/interspeech.2019-2249 · Cited by 687
Constant Q Cepstral Coefficients: A Spoofing Countermeasure for Automatic Speaker Verification
Todisco, M., Delgado, H., & Evans, N. (2017) · Computer Speech & Language · doi:10.1016/j.csl.2017.01.001 · Cited by 424 · cited 1×
“Thus, instead of us-ing the STFT, we convert time-domain audio X into a complex-valued CQT spectrogram: Previous work has already shown that magnitude-based CQT is better suited to voice antispoofing than magnitude-based STFT [24], which motivates our approach.”
MethodEnd-to-End anti-spoofing with RawNet2
Tak, H., Patino, J., Todisco, M., Nautsch, A., Evans, N., & Larcher, A. (2021) · ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp39728.2021.9414234 · Cited by 402 · cited 6×
“The second approach to voice anti-spoofing is to design models that process raw audio directly, which have been shown to be more effective [4, 5, 6], but lack transparency because ex-plainable AI methods (XAI) such as saliency maps require input data with spatial dimensions (i.e. at least…”
BackgroundMethodResultInfluentialAudio replay attack detection with deep learning frameworks
Lavrentyeva, G., Novoselov, S., Malykh, E., Kozlov, A., Kudashev, O., & Shchemelinin, V. (2017) · Interspeech 2017 · doi:10.21437/interspeech.2017-360 · Cited by 292 · cited 2×
“This dataset is widely used in related research [4, 5, 6, 9, 10, 11, 12, 13] and is considered one of the most established anti-spoofing datasets.”
MethodDoes Audio Deepfake Detection Generalize?
Müller, N., Czempin, P., Diekmann, F., Froghyar, A., & Böttinger, K. (2022) · Interspeech 2022 · doi:10.21437/interspeech.2022-108 · Cited by 258
Self-citationSTC Antispoofing Systems for the ASVspoof2019 Challenge
Lavrentyeva, G., Novoselov, S., Tseren, A., Volkova, M., Gorlanov, A., & Kozlov, A. (2019) · Interspeech 2019 · doi:10.21437/interspeech.2019-1768 · Cited by 245
Recurrent convolutional structures for audio spoof and video deepfake detection
Chintha, A., Thai, B., Sohrawardi, S. J., Bhatt, K., Hickerson, A., Wright, M., & Ptucha, R. (2020) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2020.2999185 · Cited by 226 · cited 1×
“The second approach involves directly processing the time-domain “raw” audio using either neural convolutions on the audio file itself [14] or stacks of sinc-layers [15] corresponding to rectangular band-pass filters.”
MethodDeep Residual Neural Networks for Audio Spoofing Detection
Alzantot, M., Wang, Z., & Srivastava, M. B. (2019) · Interspeech 2019 · doi:10.21437/interspeech.2019-3174 · Cited by 215
Tak, H., Jung, J.-W., Patino, J., Kamble, M., Todisco, M., & Evans, N. (2021) · 2021 Edition of the Automatic Speaker Verification and Spoofing Countermeasures Challenge · doi:10.21437/asvspoof.2021-1 · Cited by 197
A Survey of Complex-Valued Neural NetworksPreprint
Bassey, J., Qian, L., & Li, X. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2101.12249 · Cited by 196 · cited 2×
“However, more complex components such as attention and recurrence, such as GRU [25], need to be adapted, since typical activation functions like sigmoid , tanh , and softmax cannot be applied directly to complex-valued scalars [26].”
BackgroundMethodMUSDB18 - a corpus for music separation
Rafii, Z., Antoine Liutkus, A., Stöter, F.-R., Mimilakis, S. I., & Bittner, R. (2017) · HAL (Le Centre pour la Communication Scientifique Directe) · doi:10.5281/zenodo.1117371 · Cited by 175 · cited 1×
“Lastly, we add random clips of noise and music to the audio files with a probability of 5% by utilizing the MusDB18 [34] and Noise ESC 50 [35] datasets.”
MethodA Scale for the Measurement of the Psychological Magnitude Pitch
Volkmann, J., Stevens, S. S., & Newman, E. B. (1937) · The Journal of the Acoustical Society of America · doi:10.1121/1.1901999 · Cited by 170
ESC: Dataset for Environmental Sound Classification
Piczak, K. J. (2015) · Harvard Dataverse · doi:10.7910/dvn/ydeput · Cited by 156
The Effect of Silence and Dual-Band Fusion in Anti-Spoofing System
Zhang, Y., Wang, W., & Zhang, P. (2021) · Interspeech 2021 · doi:10.21437/interspeech.2021-1281 · Cited by 80 · cited 1×
“We address this issue by following the suggestions made by related work [19, 30] and trim the beginning and end silences of an audio file.”
MethodMUSDB18 - a corpus for music separation
Rafii, Z., Antoine Liutkus, A., Stöter, F.-R., et al. (2017) · Zenodo (CERN European Organization for Nuclear Research) · doi:10.5281/zenodo.1117372 · Cited by 79
Speech is Silver, Silence is Golden: What do ASVspoof-trained Models Really Learn?
Müller, N., Dieckmann, F., Czempin, P., Canals, R., Böttinger, K., & Williams, J. (2021) · 2021 Edition of the Automatic Speaker Verification and Spoofing Countermeasures Challenge · doi:10.21437/asvspoof.2021-9 · Cited by 69
Self-citationRaw Differentiable Architecture Search for Speech Deepfake and Spoofing Detection
Ge, W., Patino, J., Todisco, M., & Evans, N. (2021) · 2021 Edition of the Automatic Speaker Verification and Spoofing Countermeasures Challenge · doi:10.21437/asvspoof.2021-4 · Cited by 62 · cited 5×
“The second approach to voice anti-spoofing is to design models that process raw audio directly, which have been shown to be more effective [4, 5, 6], but lack transparency because ex-plainable AI methods (XAI) such as saliency maps require input data with spatial dimensions (i.e. at least…”
BackgroundMethodResultInfluentialWaveGrad: Estimating Gradients for Waveform Generation
Chen, N., Zhang, Y., Zen, H., Weiss, R. J., Norouzi, M., & Chan, W. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2009.00713 · Cited by 44 · cited 1×
“This phase information is crucial for audio quality and voice naturalness [1], and in the domain of speech-to-text (STT), it is laboriously recreated us-ing Griffin-Lim or neural vocoders [1, 2, 3].”
Method
Cited by
A Survey on Speech Deepfake Detection
Li, M., Ahmadiadli, Y., & Zhang, X.-P. (2025) · ACM Computing Surveys · doi:10.1145/3714458 · Cited by 101
“In the context of speech Deepfake detection, research indicates that using pre-trained wav2vec 2.0 features instead of hand-crafted features or SincNet front-ends, significantly improves detection performance when the same back-end classifier and data augmentation techniques are used [76, 104, 151].”
InfluentialMLAAD: The Multi-Language Audio Anti-Spoofing Dataset
Müller, N. M., Kawa, P., Choong, W. H., Casanova, E., Gölge, E., Müller, T., Syga, P., Sperl, P., & Böttinger, K. (2024) · 2024 International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn60899.2024.10650962 · Cited by 73
“Contrary to typical voice anti-spoofing evaluations [7], [9], [59], [65] that use the equal error rate (EER), we opt for accuracy as our evaluation metric.”
Self-citationAudio Deepfake Detection: What Has Been Achieved and What Lies Ahead
Zhang, B., Cui, H., Nguyen, V., & Whitty, M. (2025) · Sensors · doi:10.3390/s25071989 · Cited by 72
“To overcome this limitation, Müller et al. [107] proposed Complex-Valued CQT Spectrograms (C-CQTs), which retain both magnitude and phase information.”
One-Class Knowledge Distillation for Spoofing Speech Detection
Lu, J., Zhang, Y., Wang, W., Shang, Z., & Zhang, P. (2024) · ICASSP 2024 - 2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp48485.2024.10446270 · Cited by 33
“Tabel 2 shows the comparison of the results of our pro-Table ResNet[18] 16.05 Hubert+LCNN[19] 12.39 Wav2Vec XLS-128[20] 4.98 Wav2Vec+LCNN[19] 4.75 Wav2Vec+Linear[21] 3.65 Wav2Vec+AASIST[4] 2.85 OCKD(proposed) 2.27 InTheWild RawNet2[8] 33.94 CQT+CVNN[22] 26.95 Wav2Vec+Linear[21] 16.17 OCKD(proposed) 7.68 posed OCKD mod…”
InfluentialExplainable deepfake detection across different modalities: An overview of methods and challenges
Momin, M. S., Sufian, A., Barman, D., Leo, M., Distante, C., & Damer, N. (2025) · Image and Vision Computing · doi:10.1016/j.imavis.2025.105738 · Cited by 37
Metadata
- Authors
- Müller, N. M., Sperl, P., & Böttinger, K.
- Affiliations
- Fraunhofer Institute for Applied and Integrated Security (DE)
- Corresponding author
- Müller, Nicolas M.
- Venue
- INTERSPEECH 2023
- Publisher
- ISCA
- Published
- 2023-08-20
- Pages
- 3814-3818
- ISSN
- 2308-457X (ISSN-L)
- OpenAlex
- W4385823415
- Cited by
- 17
- References
- 29
- Citations by year
- 2026: 3 · 2025: 9 · 2024: 5
- FWCI
- 3.01
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- Interspeech (USB) (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- Fraunhofer-Publica (Fraunhofer-Gesellschaft) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Speech Recognition and Synthesis
- Physical Sciences › Computer Science › Signal Processing › Biometric Identification and Security
- Physical Sciences › Computer Science › Signal Processing › Speech and Audio Processing
- Keywords
- complex-valued neural networks, voice biometrics, anti-spoofing, liveness detection
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.