CoChat
Sign Up
PreprintCited by 80Speech Recognition and SynthesisDigital Media Forensic DetectionMusic and Audio Processing

WaveFake: A Data Set to Facilitate Audio Deepfake Detection

Open access
Frank, Joel, Schönherr, Lea·2021·arXiv
DOI 10.48550/arxiv.2111.02813arXiv 2111.02813
Open this paper in CoChat — free accountOpen PDF

Abstract

Deep generative modeling has the potential to cause significant harm to society. Recognizing this threat, a magnitude of research into detecting so-called "Deepfakes" has emerged. This research most often focuses on the image domain, while studies exploring generated audio signals have, so-far, been neglected. In this paper we make three key contributions to narrow this gap. First, we provide researchers with an introduction to common signal processing techniques used for analyzing audio signals. Second, we present a novel data set, for which we collected nine sample sets from five different network architectures, spanning two languages. Finally, we supply practitioners with two baseline models, adopted from the signal processing community, to facilitate further research in this area.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2111.02813

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • WaveFake dataset is generated using similar methods: 7 architectures, 6 of them GANs, some interdependent, only 4 unique.

    “We conclude that the datasets are generated using similar methods. There are 7 architectures, while 6 of them are GANs. Moreover, some methods are interdependent (e.g. MelGAN, MelGANLarge), thus only 4 of them are unique.”

    From Attack Agnostic Dataset: Towards Generalization and Stabilization of Audio DeepFake Detection (2022) · cites this paper as [18]

    Section 3 (p. 5) lists seven named model configurations, including MelGAN (L), and identifies WaveGlow as flow-based rather than GAN-based. It also describes relationships among the MelGAN variants. But the paper does not say that only four architectures are unique; its Abstract (p. 1) describes six different network architectures. (LLM, conf=0.86)

    Experimental — accuracy not yet measured

  • WaveFake dataset consists of 117,985 generated audio clips totaling 196 hours, includes both English and Japanese samples, and does not include any voice conversion algorithms.

    “The dataset (Frank and Schönherr, 2021) consists of 117,985 generated audio clips (196 h total time). However, this dataset includes both English and Japanese samples. This dataset also does not include any VC algorithms.”

    From Audio deepfakes: A survey (2023) · cites this paper as Frank and Schönherr, 2021

  • WaveFake uses various state-of-the-art text-to-speech algorithms.

    “One advantage of this dataset is that they have used various state-of-the-art text-to-speech algorithms.”

    From Audio deepfakes: A survey (2023) · cites this paper as Frank and Schönherr, 2021

  • Neural network-based deep learning models generally perform better than machine learning models.

    “While machine learning models demonstrate notable robustness, neural network-based deep learning models generally perform better [85].”

    From Audio Deepfake Detection: What Has Been Achieved and What Lies Ahead (2025) · cites this paper as [85]

  • Attack Agnostic Dataset is a combination of two DeepFake datasets (FakeAVCeleb and WaveFake) and one spoofing dataset (ASVspoof 2019 LA subset).

    “It is a combination of two DeepFake datasets - FakeAVCeleb [14] and WaveFake [18], and one dataset originally prepared for spoofing detection ASVspoof 2019 LA subset [19].”

    From Attack Agnostic Dataset: Towards Generalization and Stabilization of Audio DeepFake Detection (2022) · cites this paper as [18]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Still being checked.

Research works printed but not linked: 97

  1. [1][1] Hadi Abdullah, Kevin Warren, Vincent Bindschaedler, Nicolas Papernot, and Patrick Traynor. SoK: The Faults in our ASRs: An Overview of Attacks against Automatic Speech Recognition and Speaker Identification Systems. In IEEE Symposium on Security and Privacy (S&P) , 2020. -
  2. [2][2] Hojjat Aghakhani, Thorsten Eisenhofer, Lea Schönherr, Dorothea Kolossa, Thorsten Holz, Christopher Kruegel, and Giovanni Vigna. VENOMAVE: Clean-Label Poisoning Against Speech Recognition. Computing Research Repository (CoRR) , abs/2010.10682, 2020. - · arXiv
  3. [3][3] Muhammad Ejaz Ahmed, Il-Youp Kwak, Jun Ho Huh, Iljoo Kim, Taekkyung Oh, and Hyoungshick Kim. Void: A fast and light voice liveness detection system. In USENIX Security Symposium , 2020. -
  4. [4][4] Yang Ai and Zhen-Hua Ling. A Neural Vocoder With Hierarchical Generation of Amplitude and Phase Spectra for Statistical Parametric Speech Synthesis. IEEE/ACM Transactions on Audio, Speech, and Language Processing , 2020. -
  5. [5][5] Moustafa Alzantot, Bharathan Balaji, and Mani Srivastava. Did you hear that? Adversarial Examples Against Automatic Speech Recognition. In Advances in Neural Information Processing Systems (NeurIPS) , 2017. -
  6. [6][6] Rosana Ardila, Megan Branson, Kelly Davis, Michael Henretty, Michael Kohler, Josh Meyer, Reuben Morais, Lindsay Saunders, Francis M Tyers, and Gregor Weber. Common Voice: A Massively-Multilingual Speech Corpus. In Language Resources and Evaluation Conference , 2020. -
  7. [7][7] Sercan Arik, Gregory Diamos, Andrew Gibiansky, John Miller, Kainan Peng, Wei Ping, Jonathan Raiman, and Yanqi Zhou. Deep Voice 2: Multi-Speaker Neural Text-to-Speech. In Advances in Neural Information Processing Systems (NeurIPS) , 2017. -
  8. [8][8] Sercan Ö Arık, Mike Chrzanowski, Adam Coates, Gregory Diamos, Andrew Gibiansky, Yongguo Kang, Xian Li, John Miller, Andrew Ng, Jonathan Raiman, et al. Deep Voice: Real-Time Neural Text-to-Speech. In International Conference on Machine Learning (ICML) , 2017. -
  9. [10][10] Mikołaj Bi´ nkowski, Jeff Donahue, Sander Dieleman, Aidan Clark, Erich Elsen, Norman Casagrande, Luis C Cobo, and Karen Simonyan. High Fidelity Speech Synthesis with Adversarial Networks. In International Conference on Learning Representations (ICLR) , 2020. -
  10. [12][12] Nicholas Carlini and Hany Farid. Evading DeepFake-Image Detectors with White-and BlackBox Attacks. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , 2020. -
  11. [13][13] Nicholas Carlini and David Wagner. Audio Adversarial Examples: Targeted Attacks on Speech-to-Text. In IEEE Deep Learning and Security Workshop (DLS) , 2018. -
  12. [14][14] Nicholas Carlini, Anish Athalye, Nicolas Papernot, Wieland Brendel, Jonas Rauber, Dimitris Tsipras, Ian Goodfellow, and Aleksander Madry. On Evaluating Adversarial Robustness. Computing Research Repository (CoRR) , abs/1902.06705, 2019. - · arXiv
  13. [15][15] Nanxin Chen, Yu Zhang, Heiga Zen, Ron J Weiss, Mohammad Norouzi, and William Chan. WaveGrad: Estimating Gradients for Waveform Generation. In International Conference on Learning Representations (ICLR) , 2020. -
  14. [16][16] Akash Chintha, Bao Thai, Saniat Javid Sohrawardi, Kartavya Bhatt, Andrea Hickerson, Matthew Wright, and Raymond Ptucha. Recurrent Convolutional Structures for Audio Spoof and Video DeepFake Detection. IEEE Journal of Selected Topics in Signal Processing , 2020. -
  15. [18][18] Chris Donahue, Julian McAuley, and Miller Puckette. Adversarial Audio Synthesis. In International Conference on Learning Representations (ICLR) , 2019. -
  16. [19][19] Jeff Donahue, Sander Dieleman, Mikołaj Bi´ nkowski, Erich Elsen, and Karen Simonyan. Endto-End Adversarial Text-to-Speech. In International Conference on Learning Representations (ICLR) , 2021. -
  17. [20][20] Ricard Durall, Margret Keuper, and Janis Keuper. Watch your Up-Convolution: CNN Based Generative Deep Neural Networks are Failing to Reproduce Spectral Distributions. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , 2020. -
  18. [21][21] Cristóbal Esteban, Stephanie L Hyland, and Gunnar Rätsch. Real-Valued (Medical) Time Series Generation with Recurrent Conditional GANs. In International Conference on Learning Representations (ICLR) , 2018. -
  19. [22][22] Yuchen Fan, Yao Qian, Feng-Long Xie, and Frank K Soong. TTS Synthesis with Bidirectional LSTM Based Recurrent Neural Networks. In International Conference on Acoustics, Speech and Signal Processing (ICASSP) , 2014. -
  20. [24][24] Joel Frank, Thorsten Eisenhofer, Lea Schönherr, Asja Fischer, Dorothea Kolossa, and Thorsten Holz. Leveraging Frequency Analysis for Deep Fake Image Recognition. In International Conference on Machine Learning (ICML) , 2020. -
  21. [26][26] Ian J Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative Adversarial Networks. In Advances in Neural Information Processing Systems (NeurIPS) , 2014. -
  22. [27][27] Anmol Gulati, James Qin, Chung-Cheng Chiu, Niki Parmar, Yu Zhang, Jiahui Yu, Wei Han, Shibo Wang, Zhengdong Zhang, Yonghui Wu, et al. Conformer: Convolution-Augmented Transformer for Speech Recognition. In Proceedings of Interspeech (INTERSPEECH) , 2020. -
  23. [28][28] Karen Hao. The Biggest Threat of Deepfakes isn't the Deepfakes Themselves. MIT Technology Review , 2019. -
  24. [30][30] Tomoki Hayashi, Ryuichi Yamamoto, Katsuki Inoue, Takenori Yoshimura, Shinji Watanabe, Tomoki Toda, Kazuya Takeda, Yu Zhang, and Xu Tan. Espnet-TTS: Unified, reproducible, and integratable open source end-to-end text-to-speech toolkit. In International Conference on Acoustics, Speech and Signal Processing (ICASSP) , 2020. -
  25. [31][31] Thomas Huang, GJTGY Yang, and Greory Tang. A fast two-dimensional median filtering algorithm. IEEE transactions on acoustics, speech, and signal processing , 27(1):13-18, 1979. -
  26. [32][32] Hirofumi Inaguma, Shun Kiyono, Kevin Duh, Shigeki Karita, Nelson Yalta, Tomoki Hayashi, and Shinji Watanabe. ESPnet-ST: All-in-one speech translation toolkit. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics: System Demonstrations , 2020. -
  27. [34][34] Jee-weon Jung, Seung-bin Kim, Hye-jin Shim, Ju-ho Kim, and Ha-Jin Yu. Improved RawNet with Feature Map Scaling for Text-independent Speaker Verification using Raw Waveforms. Proceedings of Interspeech (INTERSPEECH) , 2020. -
  28. [35][35] Nal Kalchbrenner, Erich Elsen, Karen Simonyan, Seb Noury, Norman Casagrande, Edward Lockhart, Florian Stimberg, Aaron Oord, Sander Dieleman, and Koray Kavukcuoglu. Efficient Neural Audio Synthesis. In International Conference on Machine Learning (ICML) , 2018. -
  29. [36][36] Madhu R Kamble, Hemlata Tak, and Hemant A Patil. Effectiveness of Speech DemodulationBased Features for Replay Detection. In Proceedings of Interspeech (INTERSPEECH) , 2018. -
  30. [38][38] Nathan Killoran, Leo J Lee, Andrew Delong, David Duvenaud, and Brendan J Frey. Generating and designing DNA with deep generative models. arXiv preprint arXiv:1712.06148 , 2017. - · arXiv
  31. [39][39] Sungwon Kim, Sang-Gil Lee, Jongyoon Song, Jaehyeon Kim, and Sungroh Yoon. FloWaveNet: A Generative Flow for Raw Audio. In International Conference on Machine Learning (ICML) , 2019. -
  32. [40][40] Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In International Conference on Learning Representations (ICLR) , 2015. -
  33. [41][41] Diederik P Kingma and Prafulla Dhariwal. Glow: Generative Flow with Invertible 1x1 Convolutions. In Advances in Neural Information Processing Systems (NeurIPS) , 2019. -
  34. [42][42] Diederik P Kingma, Tim Salimans, Rafal Jozefowicz, Xi Chen, Ilya Sutskever, and Max Welling. Improving Variational Inference with Inverse Autoregressive Flow. In International Conference on Learning Representations (ICLR) - Workshop track , 2016. -
  35. [43][43] Tomi Kinnunen, Md Sahidullah, Mauro Falcone, Luca Costantini, Rosa González Hautamäki, Dennis Thomsen, Achintya Sarkar, Zheng-Hua Tan, Héctor Delgado, Massimiliano Todisco, Nicholas Evans, Ville Hautamäki, and Kong Aik Lee. RedDots replayed: A new replay spoofing attack corpus for text-dependent speaker verification research. In International Conference on Acoustics, Speech and Signal Processing (ICASSP) , 2017. -
  36. [44][44] Paul Kocher, Jann Horn, Anders Fogh, , Daniel Genkin, Daniel Gruss, Werner Haas, Mike Hamburg, Moritz Lipp, Stefan Mangard, Thomas Prescher, Michael Schwarz, and Yuval Yarom. Spectre attacks: Exploiting speculative execution. In 40th IEEE Symposium on Security and Privacy (S&P'19) , 2019. -
  37. [46][46] Jungil Kong, Jaehyeon Kim, and Jaekyoung Bae. Hifi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesis. In Advances in Neural Information Processing Systems (NeurIPS) , 2020. -
  38. [47][47] Zhifeng Kong, Wei Ping, Jiaji Huang, Kexin Zhao, and Bryan Catanzaro. DiffWave: A Versatile Diffusion Model for Audio Synthesis. In International Conference on Learning Representations (ICLR) , 2021. -
  39. [48][48] Kundan Kumar, Rithesh Kumar, Thibault de Boissiere, Lucas Gestin, Wei Zhen Teoh, Jose Sotelo, Alexandre de Brébisson, Yoshua Bengio, and Aaron Courville. MelGAN: Generative Adversarial Networks for Conditional Waveform Synthesis. In Advances in Neural Information Processing Systems (NeurIPS) , 2019. -
  40. [50][50] Cheng-I Lai, Alberto Abad, Korin Richmond, Junichi Yamagishi, Najim Dehak, and Simon King. Attentive Filtering Networks for Audio Replay Attack Detection. In International Conference on Acoustics, Speech and Signal Processing (ICASSP) , 2019. -
  41. [51][51] Galina Lavrentyeva, Sergey Novoselov, Egor Malykh, Alexander Kozlov, Oleg Kudashev, and Vadim Shchemelinin. Audio Replay Attack Detection with Deep Learning Frameworks. In Proceedings of Interspeech (INTERSPEECH) , 2017. -
  42. [52][52] Chenda Li, Jing Shi, Wangyou Zhang, Aswin Shanmugam Subramanian, Xuankai Chang, Naoyuki Kamo, Moto Hira, Tomoki Hayashi, Christoph Boeddeker, Zhuo Chen, and Shinji Watanabe. ESPnet-SE: End-to-end speech enhancement and separation toolkit designed for ASR integration. In Proceedings of IEEE Spoken Language Technology Workshop (SLT) , 2021. -
  43. [53][53] Yuezun Li and Siwei Lyu. Exposing DeepFake Videos by Detecting Face Warping Artifacts. arXiv preprint arXiv:1811.00656 , 2018. - · arXiv
  44. [54][54] Yuezun Li, Xin Yang, Pu Sun, Honggang Qi, and Siwei Lyu. Celeb-DF: A New Dataset for DeepFake Forensics. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2020. -
  45. [55][55] Moritz Lipp, Michael Schwarz, Daniel Gruss, Thomas Prescher, Werner Haas, Anders Fogh, Jann Horn, Stefan Mangard, Paul Kocher, Daniel Genkin, Yuval Yarom, and Mike Hamburg. Meltdown: Reading kernel memory from user space. In 27th USENIX Security Symposium (USENIX Security 18) , 2018. -
  46. [56][56] Francesco Marra, Diego Gragnaniello, Davide Cozzolino, and Luisa Verdoliva. Detection of GAN-Generated Fake Images over Social Networks. In IEEE Conference on Multimedia Information Processing and Retrieval (MIPR) , 2018. -
  47. [57][57] Francesco Marra, Diego Gragnaniello, Luisa Verdoliva, and Giovanni Poggi. Do GANs Leave Artificial Gingerprints? In IEEE Conference on Multimedia Information Processing and Retrieval (MIPR) , 2019. -
  48. [58][58] Bill McCarty. The honeynet arms race. IEEE Security & Privacy , 2003. -
  49. [59][59] Scott McCloskey and Michael Albright. Detecting GAN-Generated Imagery Using Color Cues. arXiv preprint arXiv:1812.08247 , 2018. - · arXiv
  50. [60][60] Soroush Mehri, Kundan Kumar, Ishaan Gulrajani, Rithesh Kumar, Shubham Jain, Jose Sotelo, Aaron Courville, and Yoshua Bengio. SampleRNN: An Unconditional End-to-End Neural Audio Generation Model. arXiv preprint arXiv:1612.07837 , 2016. - · arXiv
  51. [61][61] Huaxiao Mo, Bolin Chen, and Weiqi Luo. Fake Faces Identification via Convolutional Neural Network. In ACM Workshop on Information Hiding and Multimedia Security , 2018. -
  52. [64][64] Andreas Nautsch, Xin Wang, Nicholas Evans, Tomi H. Kinnunen, Ville Vestman, Massimiliano Todisco, Héctor Delgado, Md Sahidullah, Junichi Yamagishi, and Kong Aik Lee. ASVspoof 2019: Spoofing Countermeasures for the Detection of Synthesized, Converted and Replayed Speech. IEEE Transactions on Biometrics, Behavior, and Identity Science , 2021. -
  53. [65][65] Aaron van den Oord, Sander Dieleman, Heiga Zen, Karen Simonyan, Oriol Vinyals, Alex Graves, Nal Kalchbrenner, Andrew Senior, and Koray Kavukcuoglu. WaveNet: A Generative Model for Raw Audio. arXiv preprint arXiv:1609.03499 , 2016. - · arXiv
  54. [66][66] Aaron van den Oord, Oriol Vinyals, and Koray Kavukcuoglu. Neural Discrete Representation Learning. In Advances in Neural Information Processing Systems (NeurIPS) , 2017. -
  55. [67][67] Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang, Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. PyTorch: An Imperative Style, High-Performance Deep Learning Library. In Advances in Neural Information Processing Systems (NeurIPS) , 2019. -
  56. [68][68] Hemant A Patil, Madhu R Kamble, Tanvina B Patel, and Meet H Soni. Novel Variable Length Teager Energy Separation Based Instantaneous Frequency Features for Replay Detection. In Proceedings of Interspeech (INTERSPEECH) , 2017. -
  57. [69][69] Kainan Peng, Wei Ping, Zhao Song, and Kexin Zhao. Non-Autoregressive Neural Text-toSpeech. In International Conference on Machine Learning (ICML) , 2020. -
  58. [70][70] Wei Ping, Kainan Peng, and Jitong Chen. Clarinet: Parallel Wave Generation in End-to-End Text-to-Speech. In International Conference on Learning Representations (ICLR) , 2019. -
  59. [71][71] Wei Ping, Kainan Peng, Kexin Zhao, and Zhao Song. WaveFlow: A Compact Flow-based Model for Raw Audio. In International Conference on Learning Representations (ICLR) , 2020. -
  60. [75][75] Ryan Prenger, Rafael Valle, and Bryan Catanzaro. Waveglow: A Flow-based Generative Network for Speech Synthesis. In International Conference on Acoustics, Speech and Signal Processing (ICASSP) , 2019. -
  61. [76][76] Yuyang Qian, Guojun Yin, Lu Sheng, Zixuan Chen, and Jing Shao. Thinking in Frequency: Face Forgery Detection by Mining Frequency-Aware Clues. In European Conference on Computer Vision (ECCV) , 2020. -
  62. [77][77] Thomas Quatieri. Discrete-Time Speech Signal Processing: Principles and Practice . Pearson Education India, 2006. -
  63. [79][79] Yi Ren, Yangjun Ruan, Xu Tan, Tao Qin, Sheng Zhao, Zhou Zhao, and Tie-Yan Liu. FastSpeech: Fast, Robust and Controllable Text to Speech. In Advances in Neural Information Processing Systems (NeurIPS) , 2019. -
  64. [80][80] Yi Ren, Chenxu Hu, Xu Tan, Tao Qin, Sheng Zhao, Zhou Zhao, and Tie-Yan Liu. FastSpeech 2: Fast and High-Quality End-to-End Text to Speech. arXiv preprint arXiv:2006.04558 , 2020. - · arXiv
  65. [81][81] Vincent Rijmen and Joan Daemen. Advanced Encryption Standard. Proceedings of Federal Information Processing Standards Publications, National Institute of Standards and Technology , 2001. -
  66. [82][82] Andreas Rossler, Davide Cozzolino, Luisa Verdoliva, Christian Riess, Justus Thies, and Matthias Nießner. Faceforensics++: Learning to Detect Manipulated Facial Images. In Proceedings of the IEEE/CVF International Conference on Computer Vision , 2019. -
  67. [83][83] Md Sahidullah, Tomi Kinnunen, and Cemal Hanilçi. A Comparison of Features for Synthetic Speech Detection. In Proceedings of Interspeech (INTERSPEECH) , 2015. -
  68. [86][86] Lea Schönherr, Steffen Zeiler, and Dorothea Kolossa. Spoofing Detection via Simultaneous Verification of Audio-Visual Synchronicity and Transcription. In 2017 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU) , 2017. -
  69. [87][87] Lea Schönherr, Katharina Kohls, Steffen Zeiler, Thorsten Holz, and Dorothea Kolossa. Adversarial Attacks Against Automatic Speech Recognition Systems via Psychoacoustic Hiding. In Symposium on Network and Distributed System Security (NDSS) , 2019. -
  70. [88][88] Lea Schönherr, Thorsten Eisenhofer, Steffen Zeiler, Thorsten Holz, and Dorothea Kolossa. Imperio: Robust Over-the-Air Adversarial Examples for Automatic Speech Recognition Systems. In Annual Computer Security Applications Conference (ACSAC) , 2020. -
  71. [89][89] Claude E Shannon. Communication Theory of Secrecy Systems. The Bell system technical journal , 1949. -
  72. [90][90] Jonathan Shen, Ruoming Pang, Ron J Weiss, Mike Schuster, Navdeep Jaitly, Zongheng Yang, Zhifeng Chen, Yu Zhang, Yuxuan Wang, Rj Skerrv-Ryan, et al. Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictions. In International Conference on Acoustics, Speech and Signal Processing (ICASSP) , 2018. -
  73. [91][91] Robin Sommer and Vern Paxson. Outside the Closed World: On Using Machine Learning for Network Intrusion Detection. In IEEE Symposium on Security and Privacy (S&P) , 2010. -
  74. [92][92] Ryosuke Sonobe, Shinnosuke Takamichi, and Hiroshi Saruwatari. JSUT Corpus: Free Large-Scale Japanese Speech Corpus for End-to-End Speech Synthesis. arXiv preprint arXiv:1711.00354 , 2017. - · arXiv
  75. [93][93] Jose Sotelo, Soroush Mehri, Kundan Kumar, Joao Felipe Santos, Kyle Kastner, Aaron Courville, and Yoshua Bengio. Char2wav: End-to-End Speech Synthesis. In International Conference on Learning Representations (ICLR) Workshop Track , 2017. -
  76. [94][94] Catherine Stupp. Fraudsters Used AI to Mimic CEO's Voice in Unusual Cybercrime Case. The Wall Street Journal , 2019. -
  77. [95][95] Yaniv Taigman, Lior Wolf, Adam Polyak, and Eliya Nachmani. VoiceLoop: Voice Fitting and Synthesis via a Phonological Loop. In International Conference on Learning Representations (ICLR) , 2017. -
  78. [96][96] Hemlata Tak, Jose Patino, Massimiliano Todisco, Andreas Nautsch, Nicholas Evans, and Anthony Larcher. End-to-End anti-spoofing with RawNet2. In International Conference on Acoustics, Speech and Signal Processing (ICASSP) , 2021. -
  79. [97][97] Shahroz Tariq, Sangyup Lee, Hoyoung Kim, Youjin Shin, and Simon S Woo. GAN is a Friend or Foe? A Framework to Detect Various Fake Face Images. In ACM/SIGAPP Symposium on Applied Computing , 2019. -
  80. [98][98] Massimiliano Todisco, Xin Wang, Ville Vestman, Md Sahidullah, Hector Delgado, Andreas Nautsch, Junichi Yamagishi, Nicholas Evans, Tomi Kinnunen, and Kong Aik Lee. ASVspoof 2019: Future Horizons in Spoofed and Fake Audio Detection. Computing Research Repository (CoRR) , abs/1904.05441, 2019. - · arXiv
  81. [99][99] Keiichi Tokuda, Takayoshi Yoshimura, Takashi Masuko, Takao Kobayashi, and Tadashi Kitamura. Speech Parameter Generation Algorithms for HMM-Based Speech Synthesis. In International Conference on Acoustics, Speech and Signal Processing (ICASSP) , 2000. -
  82. [100][100] Francis Tom, Mohit Jain, and Prasenjit Dey. End-To-End Audio Replay Attack Detection Using Deep Convolutional Networks with Attention. In Proceedings of Interspeech (INTERSPEECH) , 2018. -
  83. [101][101] Rafael Valle, Wilson Cai, and Anish Doshi. TequilaGAN: How to Easily Identify GAN Samples. arXiv preprint arXiv:1807.04919 , 2018. - · arXiv
  84. [102][102] Sheng-Yu Wang, Oliver Wang, Richard Zhang, Andrew Owens, and Alexei A Efros. CNNgenerated images are surprisingly easy to spot... for now. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , 2020. -
  85. [103][103] Xin Wang, Shinji Takaki, and Junichi Yamagishi. Neural Source-Filter Waveform Models for Statistical Parametric Speech Synthesis. IEEE/ACM Transactions on Audio, Speech, and Language Processing , 2019. -
  86. [104][104] Shinji Watanabe, Takaaki Hori, Shigeki Karita, Tomoki Hayashi, Jiro Nishitoba, Yuya Unno, Nelson Enrique Yalta Soplin, Jahn Heymann, Matthew Wiesner, Nanxin Chen, Adithya Renduchintala, and Tsubasa Ochiai. ESPnet: End-to-end speech processing toolkit. In Proceedings of Interspeech (INTERSPEECH) , 2018. -
  87. [106][106] Zhizheng Wu, Junichi Yamagishi, Tomi Kinnunen, Cemal Hanilçi, Mohammed Sahidullah, Aleksandr Sizov, Nicholas Evans, Massimiliano Todisco, and Héctor Delgado. ASVspoof: The Automatic Speaker Verification Spoofing and Countermeasures Challenge. IEEE Journal of Selected Topics in Signal Processing , 2017. -
  88. [107][107] Shawn Xu, Subhashini Venugopalan, and Mukund Sundararajan. Attribution in Scale and Space. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , 2020. -
  89. [108][108] Ryuichi Yamamoto, Eunwoo Song, and Jae-Min Kim. Parallel WaveGAN: A Fast Waveform Generation Model Based on Generative Adversarial Networks with Multi-Resolution Spectrogram. In International Conference on Acoustics, Speech and Signal Processing (ICASSP) , 2020. -
  90. [109][109] Geng Yang, Shan Yang, Kai Liu, Peng Fang, Wei Chen, and Lei Xie. Multi-Band Melgan: Faster Waveform Generation For High-Quality Text-To-Speech. In Proceedings of IEEE Spoken Language Technology Workshop (SLT) , 2021. -
  91. [110][110] Takayoshi Yoshimura, Keiichi Tokuda, Takashi Masuko, Takao Kobayashi, and Tadashi Kitamura. Simultaneous Modeling of Spectrum, Pitch and Duration in HMM-Based Speech Synthesis. In Sixth European Conference on Speech Communication and Technology , 1999. -
  92. [111][111] Ning Yu, Larry S Davis, and Mario Fritz. Attributing Fake Images to GANs: Learning and Analyzing GAN Fingerprints. In IEEE International Conference on Computer Vision (ICCV) , 2019. -
  93. [112][112] Xuejing Yuan, Yuxuan Chen, Yue Zhao, Yunhui Long, Xiaokang Liu, Kai Chen, Shengzhi Zhang, Heqing Huang, Xiaofeng Wang, and Carl A. Gunter. CommanderSong: A Systematic Approach for Practical Adversarial Voice Recognition. In USENIX Security Symposium , 2018. -
  94. [113][113] Heiga Ze, Andrew Senior, and Mike Schuster. Statistical Parametric Speech Synthesis Using Deep Neural Networks. In International Conference on Acoustics, Speech and Signal Processing (ICASSP) , 2013. -
  95. [115][115] Linghan Zhang, Sheng Tan, Jie Yang, and Yingying Chen. Voicelive: A Phoneme Localization Based Liveness Detection for Voice Authentication on Smartphones. In ACM Conference on Computer and Communications Security (CCS) , 2016. -
  96. [116][116] Xu Zhang, Svebor Karaman, and Shih-Fu Chang. Detecting and Simulating Artifacts in GAN Fake Images. In 2019 IEEE International Workshop on Information Forensics and Security (WIFS) , 2019. -
  97. [117][117] Eberhard Zwicker and Hugo Fastl. Psychoacoustics: Facts and Models . Springer, Heidelberg, Germany, 2 edition, 2007.

Cited by

Sorted by citations

Related

Authors
Frank, J., & Schönherr, L.
Affiliations
Ruhr University Bochum (DE)
ORCID
Schönherr, Lea
Venue
arXiv
Publisher
arXiv
Published
2021
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.2111.02813
arXiv
2111.02813
OpenAlex
W4226246616
Cited by
76
References
0
Citations by year
2026: 1 · 2025: 17 · 2024: 34 · 2023: 17 · 2022: 7
Type
posted content
Language
en
Open access
Licence
creativecommons.org/licenses/by-sa/4.0/legalcode
Locations
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Funders
Deutsche Forschungsgemeinschaft (390781972)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Speech Recognition and Synthesis
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Digital Media Forensic Detection
  • Physical Sciences › Computer Science › Signal Processing › Music and Audio Processing
Keywords
audio deepfake detection, deepfakes, synthetic audio, audio signal processing, generative audio models, audio forensics

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review