Fake Speech Detection Using Residual Network with Transformer Encoder
The DOI matches its Crossref record; 28 of the 28 references it cites were found in a registry or an index.
Abstract
Fake speech detection aims to distinguish fake speech from natural speech. This paper presents an effective fake speech detection scheme based on residual network with transformer encoder (TE-ResNet) for improving the performance of fake speech detection. Firstly, considering inter-frame correlation of the speech signal, we utilize transformer encoder to extract contextual representations of the acoustic features. Then, a residual network is used to process deep features and calculate score that the speech is fake. Besides, to increase the quantity of training data, we apply five speech data augmentation techniques on the training dataset. Finally, we fuse the different fake speech detection models on score-level by logistic regression for compensating the shortcomings of each single model. The proposed scheme is evaluated on two public speech datasets. Our experiments demonstrate that the proposed TE-ResNet outperforms the existing state-of-the-art methods both on development and evaluation datasets. In addition, the proposed fused model achieves improved performance for detection of unseen fake speech technology, which can obtain equal error rates (EERs) of 3.99% and 5.89% on evaluation set of FoR-normal dataset and ASVspoof 2019 LA dataset respectively.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Zhang et al. [110] utilised MFCC as a frontend feature in their TE-ResNet framework, achieving near-optimal detection performance.”
From Audio Deepfake Detection: What Has Been Achieved and What Lies Ahead (2025) · cites this paper as [110]
In the citing paper
“Zhang et al. [110] explored CQCC alongside MFCC, revealing their significant advantages in multiresolution analysis. CQCC demonstrated superior performance in capturing complex spectral structures and detecting deepfake artefacts”
From Audio Deepfake Detection: What Has Been Achieved and What Lies Ahead (2025) · cites this paper as [110]
In the citing paper
“Another study [110] incorporated Transformer Encoders (TEs) to extract contextual representations of acoustic features, which were then fed into ResNet. The approach demonstrated remarkable performance across multiple datasets, outperforming CNNs and LCNN in terms of both accuracy and generalisation.”
From Audio Deepfake Detection: What Has Been Achieved and What Lies Ahead (2025) · cites this paper as [110]
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
The meaning and use of the area under a receiver operating characteristic (ROC) curve.
Hanley, J. A., & McNeil, B. J. (1982) · Radiology · doi:10.1148/radiology.143.1.7063747 · Cited by 21,856 · cited 1×
“In order to view the performance of detectors for detecting the fake speech from another perspective, we also use receiver operating characteristic (ROC) curve [6] as a secondary metric.”
MethodProceedings of the 21st International Conference on Neural Information Processing Systems
Koller, D., Schuurmans, D., Bengio, Y., & Bottou, L. (2008) · Cited by 4,461
ASVspoof 2019: Future Horizons in Spoofed and Fake Audio Detection
Todisco, M., Wang, X., Vestman, V., Sahidullah, M., Delgado, H., Nautsch, A., Yamagishi, J., Evans, N., Kinnunen, T., & Lee, K. A. (2019) · arXiv · doi:10.48550/arxiv.1904.05441 · Cited by 688
ASVspoof 2019: Future Horizons in Spoofed and Fake Audio Detection
Todisco, M., Wang, X., Vestman, V., Sahidullah, M., Delgado, H., Nautsch, A., Yamagishi, J., Evans, N., Kinnunen, T. H., & Lee, K. A. (2019) · Interspeech 2019 · doi:10.21437/interspeech.2019-2249 · Cited by 687
ASVspoof 2019: A large-scale public database of synthesized, converted and replayed speech
Wang, X., Yamagishi, J., Todisco, M., Delgado, H., Nautsch, A., Evans, N., Sahidullah, M., Vestman, V., Kinnunen, T., Lee, K. A., et al. (2020) · Computer Speech & Language · doi:10.1016/j.csl.2020.101114 · Cited by 505 · cited 1×
“The organizers of ASVspoof challenge have published three editions of databases, the last edition of ASVspoof 2019 database varied coverage of different types of spoofing data, could further foster research on anti-spoofing [29].”
BackgroundConstant Q Cepstral Coefficients: A Spoofing Countermeasure for Automatic Speaker Verification
Todisco, M., Delgado, H., & Evans, N. (2017) · Computer Speech & Language · doi:10.1016/j.csl.2017.01.001 · Cited by 424
A comparison of features for synthetic speech detection
Sahidullah, M., Kinnunen, T., & Hanilçi, C. (2015) · Interspeech 2015 · doi:10.21437/interspeech.2015-472 · Cited by 382 · cited 1×
“[20] made a comparative study and demonstrated the efficacy of various short-term power spectrum and phase features for synthetic speech detection.”
BackgroundA New Feature for Automatic Speaker Verification Anti-Spoofing: Constant Q Cepstral Coefficients
Todisco, M., Delgado, H., & Evans, N. (2016) · The Speaker and Language Recognition Workshop (Odyssey 2016) · doi:10.21437/odyssey.2016-41 · Cited by 335
Audio replay attack detection with deep learning frameworks
Lavrentyeva, G., Novoselov, S., Malykh, E., Kozlov, A., Kudashev, O., & Shchemelinin, V. (2017) · Interspeech 2017 · doi:10.21437/interspeech.2017-360 · Cited by 292
STC Antispoofing Systems for the ASVspoof2019 Challenge
Lavrentyeva, G., Novoselov, S., Tseren, A., Volkova, M., Gorlanov, A., & Kozlov, A. (2019) · Interspeech 2019 · doi:10.21437/interspeech.2019-1768 · Cited by 245
ASVspoof: The Automatic Speaker Verification Spoofing and Countermeasures Challenge
Wu, Z., Yamagishi, J., Kinnunen, T., Hanilci, C., Sahidullah, M., Sizov, A., Evans, N., Todisco, M., & Delgado, H. (2017) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2017.2671435 · Cited by 229
Deep Residual Neural Networks for Audio Spoofing Detection
Alzantot, M., Wang, Z., & Srivastava, M. B. (2019) · Interspeech 2019 · doi:10.21437/interspeech.2019-3174 · Cited by 215
FoR: A Dataset for Synthetic Speech Detection
Reimao, R., & Tzerpos, V. (2019) · 2019 International Conference on Speech Technology and Human-Computer Dialogue (SpeD) · doi:10.1109/sped.2019.8906599 · Cited by 193
A Comparison on Data Augmentation Methods Based on Deep Learning for Audio Classification
Wei, S., Zou, S., Liao, F., & Lang, W. (2020) · Journal of Physics Conference Series · doi:10.1088/1742-6596/1453/1/012085 · Cited by 176
Deep Multi-Model Fusion for Single-Image Dehazing
Deng, Z., Zhu, L., Hu, X., Fu, C., Xu, X., Zhang, Q., Qin, J., & Heng, P. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00254 · Cited by 162
Advances in anti-spoofing: from the perspective of ASVspoof challenges
Kamble, M. R., Sailor, H. B., Patil, H. A., & Li, H. (2020) · APSIPA Transactions on Signal and Information Processing · doi:10.1017/atsip.2019.21 · Cited by 102
Improving Sequence-To-Sequence Speech Recognition Training with On-The-Fly Data Augmentation
Nguyen, T.-S., Stuker, S., Niehues, J., & Waibel, A. (2020) · ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp40776.2020.9054130 · Cited by 90
Spectral Features for Synthetic Speech Detection
Paul, D., Pal, M., & Saha, G. (2017) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2017.2684705 · Cited by 77
The Attacker’s Perspective on Automatic Speaker Verification: An Overview
Das, R. K., Tian, X., Kinnunen, T., & Li, H. (2020) · Interspeech 2020 · doi:10.21437/interspeech.2020-1052 · Cited by 76
Joint Speaker Verification and Antispoofing in the $i$ -Vector Space
Sizov, A., Khoury, E., Kinnunen, T., Wu, Z., & Marcel, S. (2015) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2015.2407362 · Cited by 74
Robust deep feature for spoofing detection — the SJTU system for ASVspoof 2015 challenge
Chen, N., Qian, Y., Dinkel, H., Chen, B., & Yu, K. (2015) · Interspeech 2015 · doi:10.21437/interspeech.2015-474 · Cited by 69
Spoofing Attack Detection Using the Non-Linear Fusion of Sub-Band Classifiers
Tak, H., Patino, J., Nautsch, A., Evans, N., & Todisco, M. (2020) · Interspeech 2020 · doi:10.21437/interspeech.2020-1844 · Cited by 65
Synthetic speech detection using fundamental frequency variation and spectral features
Pal, M., Paul, D., & Saha, G. (2017) · Computer Speech & Language · doi:10.1016/j.csl.2017.10.001 · Cited by 59
Synthetic Speech Detection Using Neural Networks
Reimao, R., & Tzerpos, V. (2021) · 2021 International Conference on Speech Technology and Human-Computer Dialogue (SpeD) · doi:10.1109/sped53181.2021.9587406 · Cited by 19
Trends in the Regulation of Hate Speech and Fake News: A Threat to Free Speech?
Santuraki, S. U. (2019) · Hasanuddin Law Review · doi:10.20956/halrev.v5i2.1625 · Cited by 15
Synthesized Speech Detection Based on Spectrogram and Convolutional Neural Networks
Nosek, T., Suzić, S., Papic, B., & Jakovljevic, N. M. (2019) · Telecommunications Forum (TELFOR) · doi:10.1109/telfor48224.2019.8971215 · Cited by 11
Synthetic Speech Spoofing Detection using MFCC and SVM
Sonawane, A., & Inamdar, M. (2017) · International journal of advance research, ideas and innovations in technology · Cited by 3
Research works printed but not linked: 5
- Bhusan Chettri , Daniel Stoller , Veronica Morfi , Marco A Mart'inez Ram'irez , Emmanouil Benetos, and Bob L Sturm. 2019 . Ensemble models for spoofing detection in automatic speaker verification. arXiv preprint arXiv:1904.04589 (2019). Bhusan Chettri, Daniel Stoller, Veronica Morfi, Marco A Mart'inez Ram'irez, Emmanouil Benetos, and Bob L Sturm. 2019. Ensemble models for spoofing detection in automatic speaker verification. arXiv preprint arXiv:1904.04589 (2019). · arXiv
- Jee-weon Jung, Hye-jin Shim, Hee-Soo Heo , and Ha-Jin Yu. 2019. Replay attack detection with complementary high-resolution information using end-to-end DNN for the ASVspoof 2019 Challenge. arXiv preprint arXiv:1904.10134 ( 2019 ). Jee-weon Jung, Hye-jin Shim, Hee-Soo Heo, and Ha-Jin Yu. 2019. Replay attack detection with complementary high-resolution information using end-to-end DNN for the ASVspoof 2019 Challenge. arXiv preprint arXiv:1904.10134 (2019). · arXiv
- Diederik P Kingma and Jimmy Ba . 2014 . Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014). Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014). · arXiv
- Alan V Oppenheim , John R Buck , and Ronald W Schafer . 2001. Discrete-time signal processing .Upper Saddle River , NJ : Prentice Hall . Alan V Oppenheim, John R Buck, and Ronald W Schafer. 2001. Discrete-time signal processing .Upper Saddle River, NJ: Prentice Hall.
- Zhizheng Wu , Tomi Kinnunen , Eng Siong Chng , Haizhou Li , and Eliathamby Ambikairajah . 2012 . A study on spoofing attack in state-of-the-art speaker verification: The telephone speech case . In Proceedings of The 2012 Asia Pacific Signal and Information Processing Association Annual Summit and Conference. IEEE, 1--5. Zhizheng Wu, Tomi Kinnunen, Eng Siong Chng, Haizhou Li, and Eliathamby Ambikairajah. 2012. A study on spoofing attack in state-of-the-art speaker verification: The telephone speech case. In Proceedings of The 2012 Asia Pacific Signal and Information Processing Association Annual Summit and Conference. IEEE, 1--5.
Cited by
Masood, M., Nawaz, M., Malik, K. M., Javed, A., Irtaza, A., & Malik, H. (2023) · Applied Intelligence · doi:10.1007/s10489-022-03766-z · Cited by 543
Does Audio Deepfake Detection Generalize?
Müller, N., Czempin, P., Diekmann, F., Froghyar, A., & Böttinger, K. (2022) · Interspeech 2022 · doi:10.21437/interspeech.2022-108 · Cited by 258
A Survey on the Detection and Impacts of Deepfakes in Visual, Audio, and Textual Formats
Mubarak, R., Alsboui, T., Alshaikh, O., Inuwa-Dutse, I., Khan, S., & Parkinson, S. (2023) · IEEE Access · doi:10.1109/access.2023.3344653 · Cited by 132
A Survey on Speech Deepfake Detection
Li, M., Ahmadiadli, Y., & Zhang, X.-P. (2025) · ACM Computing Surveys · doi:10.1145/3714458 · Cited by 101
Shaaban, O. A., Yıldırım, R., & Alguttar, A. A. (2023) · IEEE Access · doi:10.1109/access.2023.3333866 · Cited by 107
Metadata
- Authors
- Zhang, Z., Yi, X., & Zhao, X.
- Affiliations
- Chinese Academy of Sciences (CN)
- Venue
- Proceedings of the 2021 ACM Workshop on Information Hiding and Multimedia Security
- Publisher
- ACM
- Published
- 2021-06-17
- Pages
- 13-22
- OpenAlex
- W3175036090
- Cited by
- 66
- References
- 27
- Citations by year
- 2026: 6 · 2025: 20 · 2024: 17 · 2023: 11 · 2022: 12
- FWCI
- 4.62
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- ACM Workshop on Information Hiding and Multimedia Security (IH&MMSec) (publishedVersion)
- Funders
- NSFC (61902391, 61972390, U1736214, 61802393 and 61872356)
- IIE CAS Climbing Program
- National Key Technology R&D Program (2019QY2202, 2019QY(Y)0207 and 2020AAA0140000)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Speech Recognition and Synthesis
- Physical Sciences › Computer Science › Signal Processing › Speech and Audio Processing
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Digital Media Forensic Detection
- Keywords
- Transformer encoder, ResNet, score-level fusion, inter-frame correlation, acoustic features
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.