Recurrent Convolutional Structures for Audio Spoof and Video Deepfake Detection
The DOI matches its Crossref record; 80 of the 80 references it cites were found in a registry or an index.
Abstract
Deepfakes, or artificially generated audiovisual renderings, can be used to defame a public figure or influence public opinion. With the recent discovery of generative adversarial networks, an attacker using a normal desktop computer fitted with an off-the-shelf graphics processing unit can make renditions realistic enough to easily fool a human observer. Detecting deepfakes is thus becoming important for reporters, social media platforms, and the general public. In this work, we introduce simple, yet surprisingly efficient digital forensic methods for audio spoof and visual deepfake detection. Our methods combine convolutional latent representations with bidirectional recurrent structures and entropy-based cost functions. The latent representations for both audio and video are carefully chosen to extract semantically rich information from the recordings. By feeding these into a recurrent framework, we can detect both spatial and temporal signatures of deepfake renditions. The entropy-based cost functions work well in isolation as well as in context with traditional cost functions. We demonstrate our methods on the FaceForensics++ and Celeb-DF video datasets and the ASVSpoof 2019 Logical Access audio datasets, achieving new benchmarks in all categories. We also perform extensive studies to demonstrate generalization to new domains and gain further insight into the effectiveness of the new architectures.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“The ASVSpoof 2019 challenge dataset illustrates the resilience of their techniques for audio spoof detection against compression and out-of-sample inference.”
From Deepfake detection using deep learning methods: A systematic and comprehensive review (2024) · cites this paper as Chintha et al. (2020)
In the citing paper
“They show the robustness of their techniques for visual deepfake identification to both compression and out-of-sample inference on the prominent FaceForensics ++ and Celeb-DF datasets, as well as the newly released deepfake detection challenge.”
From Deepfake detection using deep learning methods: A systematic and comprehensive review (2024) · cites this paper as Chintha et al. (2020)
In the citing paper
“Their approaches achieve new benchmark standards for deepfake visual and spoof audio detection and show that they generalize well to unexpected attacks.”
From Deepfake detection using deep learning methods: A systematic and comprehensive review (2024) · cites this paper as Chintha et al. (2020)
In the citing paper
“a voice deepfake was used to scam a CEO out of $243,000 [22].”
From Deep learning for deepfakes creation and detection: A survey (2022) · cites this paper as [22]
In the citing paper
“Existing unimodal deepfake detectors are primarily designed to detect a single type of manipulation, such as visual, acoustic, and text.”
From Contextual Cross-Modal Attention for Audio-Visual Deepfake Detection and Localization (2024) · cites this paper as [13]
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · cited 2×
“Thenetworkarchitecturenotonlyuses Thenetworkarchitecturenotonlyuses skipconnectionsakintoResNet[45]withanInception-like[46] convolutionalarrangement,butitalsohasamodifiedversion ofdepth-wiseseparableconvolutionallayersforreducingthe numberofparameterswithmarginallybetterperformance.”
BackgroundVery Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946 · cited 1×
“The optimizer is Adam, with β 1 set to 0.9 and β 2 set to 0.999, these being the default values suggested by Kingma and Ba’s original paper on Adam [74].”
MethodInfluentialImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
Densely Connected Convolutional Networks
Huang, G., Liu, Z., Van Der Maaten, L., & Weinberger, K. Q. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.243 · Cited by 48,566
Rethinking the Inception Architecture for Computer Vision
Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., & Wojna, Z. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.308 · Cited by 32,138
Unpaired image-to-image translation using cycle-consistent adversarial networks
Zhu, J.-Y., Park, T., Isola, P., & Efros, A. A. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.244 · Cited by 23,602
Xception: Deep Learning with Depthwise Separable Convolutions
Chollet, F. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.195 · Cited by 20,380 · cited 5×
“Our architecture for deepfake detection is inspired by the XceptionNet [32] architecture along with recurrent processing used in ConvLSTM [20] and FaceNetLSTM [33].”
BackgroundMethodInfluentialAuto-Encoding Variational Bayes
Kingma, D. P., & Welling, M. (2013) · arXiv · doi:10.48550/arxiv.1312.6114 · Cited by 20,067 · cited 1×
“Inspired from the application of KL divergence in variational autoencoders [67], the high-level idea is to disentangle different probability distributions via parameter learning.”
MethodLabaca-Castro, R. (2023) · Machine Learning under Malware Attack · doi:10.1007/978-3-658-40442-0_9 · Cited by 19,272
FaceNet: A unified embedding for face recognition and clustering
Schroff, F., Kalenichenko, D., & Philbin, J. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298682 · Cited by 15,267 · cited 4×
“In a short paper, Sohrawardi et al. [33] described and very briefly evaluated models based on the FaceNet [50] CNN architecture, with weights initialized from the VGGFace2 [51] facial recognition task.”
MethodInfluentialAggregated Residual Transformations for Deep Neural Networks
Xie, S., Girshick, R., Dollar, P., Tu, Z., & He, K. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.634 · Cited by 12,465 · cited 1×
“The architecture of the WideBlock, taking in-spiration from ResNeXt blocks used in image classification [77], consists of several parallel streams, each consisting of bottleneck 1 × 1 convolution layers before and after a normal convolution layer.”
MethodUnsupervised representation learning with deep convolutional generative adversarial networks
Radford, A., Metz, L., & Chintala, S. (2015) · arXiv · doi:10.48550/arxiv.1511.06434 · Cited by 12,235
A Style-Based Generator Architecture for Generative Adversarial Networks
Karras, T., Laine, S., & Aila, T. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00453 · Cited by 11,714
WaveNet: a generative model for raw audio
Oord, A. V. D., Dieleman, S., Zen, H., Simonyan, K., Vinyals, O., Graves, A., Kalchbrenner, N., Senior, A., & Kavukcuoglu, K. (2016) · arXiv · doi:10.48550/arxiv.1609.03499 · Cited by 4,911 · cited 1×
“…LAneedonlyafewminutesofrecordedaudioofatargetspeaker tocreateaccuratespoofsthatarehardtodetectbyhumans. van den Oord et al. [52] introduced WaveNet, a deep neural network model that can generate speech from a text in a fully probabilistic and autoregressive manner and can…”
BackgroundProgressive Growing of GANs for Improved Quality, Stability, and Variation
Karras, T., Aila, T., Laine, S., & Lehtinen, J. (2017) · arXiv · doi:10.48550/arxiv.1710.10196 · Cited by 4,727
Speaker Verification Using Adapted Gaussian Mixture Models
Reynolds, D. A., Quatieri, T. F., & Dunn, R. B. (2000) · Digital Signal Processing · doi:10.1006/dspr.1999.0361 · Cited by 4,268 · cited 1×
“such as constant-Q cepstrum coefficients (CQCC) [57], or MFCCs, have been combined with Gaussian Mixture Models (GMMs) [58] and deep neural networks.”
MethodDynamic Routing Between Capsules
Sabour, S., Frosst, N., & Hinton, G. E. (2017) · arXiv · doi:10.48550/arxiv.1710.09829 · Cited by 4,167
StarGAN: Unified Generative Adversarial Networks for Multi-domain Image-to-Image Translation
Choi, Y., Choi, M., Kim, M., Ha, J.-W., Kim, S., & Choo, J. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00916 · Cited by 4,163
FaceForensics++: Learning to Detect Manipulated Facial Images
Rossler, A., Cozzolino, D., Verdoliva, L., Riess, C., Thies, J., & Niessner, M. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00009 · Cited by 3,314 · cited 9×
“TABLE III WITHIN-DOMAIN RESULTS: VIDEO-LEVEL ACCURACY ON FACEFORENSICS++ [22] AND CELEB-DF [69] OFFICIAL TEST SETS.”
BackgroundMethodInfluentialVGGFace2: A Dataset for Recognising Faces across Pose and Age
Cao, Q., Shen, L., Xie, W., Parkhi, O. M., & Zisserman, A. (2018) · 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018) · doi:10.1109/fg.2018.00020 · Cited by 3,032
Dlib-ml: A Machine Learning Toolkit
King, D. E. (2009) · doi:10.5555/1577069.1755843 · Cited by 2,777 · cited 1×
“The dlib [66] face detector determines the primary face over each frame in the video.”
BackgroundStructural optimization using sensitivity analysis and a level-set method
Allaire, G., Jouve, F., & Toader, A.-M. (2004) · Journal of Computational Physics · doi:10.1016/j.jcp.2003.09.032 · Cited by 2,759 · cited 1×
“We hypothesize that the probability distribution of the real and ersatz material [68] when mapped into a two-dimensional space from the latent feature representation generated after the primary fully-connected layer can be disentangled.”
BackgroundGenerative Adversarial Text to Image Synthesis
Reed, S., Akata, Z., Yan, X., Logeswaran, L., Schiele, B., & Lee, H. (2016) · arXiv · doi:10.48550/arxiv.1605.05396 · Cited by 2,494 · cited 1×
“To promote variation, G is seeded with a noise vector, but this noise vector can be paired with a latent representation of an object (word, image, sentence), to constrain the resulting image [35].”
BackgroundSignal estimation from modified short-time Fourier transform
Griffin, D., & Lim, J. (1984) · IEEE Transactions on Acoustics, Speech, and Signal Processing · doi:10.1109/tassp.1984.1164317 · Cited by 2,095
MesoNet: a Compact Facial Video Forgery Detection Network
Afchar, D., Nozick, V., Yamagishi, J., & Echizen, I. (2018) · 2018 IEEE International Workshop on Information Forensics and Security (WIFS) · doi:10.1109/wifs.2018.8630761 · Cited by 1,872
Learning to Discover Cross-Domain Relations with Generative Adversarial Networks
Kim, T., Cha, M., Kim, H., Lee, J. K., & Kim, J. (2017) · arXiv · doi:10.48550/arxiv.1703.05192 · Cited by 1,716 · cited 1×
“Available: https://www.buzzfeednews.com/article/davidmack/ obama-fake-news-jordan-peele-psa-video-buzzfeed Zhu et al. [36] and Kim et al. [37] built on the GAN concept and replaced the noise vector with input images.”
BackgroundHinton, G. E., Krizhevsky, A., & Wang, S. D. (2011) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-21735-7_6 · Cited by 1,360 · cited 1×
“The Capsule-Forensics method proposed by Nguyen et al. [21] uses capsule networks [42] for the detection of replay attacks as well as computer-generated images and videos.”
MethodDeepfake Video Detection Using Recurrent Neural Networks
Guera, D., & Delp, E. J. (2018) · 2018 15th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS) · doi:10.1109/avss.2018.8639163 · Cited by 1,285 · cited 7×
“These features were then passed into a ConvLSTM network [20].”
BackgroundMethodResultInfluentialExposing Deep Fakes Using Inconsistent Head Poses
Yang, X., Li, Y., & Lyu, S. (2019) · ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2019.8683164 · Cited by 1,196
Jointly Embedding Multiple Single-Cell Omics Measurements
Liu, J., Huang, Y., Singh, R., Vert, J., & Noble, W. S. (2019) · Leibniz international proceedings in informatics · doi:10.4230/lipics.wabi.2019.10 · Cited by 1,141
In Ictu Oculi: Exposing AI Created Fake Videos by Detecting Eye Blinking
Li, Y., Chang, M.-C., & Lyu, S. (2018) · 2018 IEEE International Workshop on Information Forensics and Security (WIFS) · doi:10.1109/wifs.2018.8630787 · Cited by 1,089
Interpreting the Latent Space of GANs for Semantic Face Editing
Shen, Y., Gu, J., Tang, X., & Zhou, B. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00926 · Cited by 1,063
Synthesizing Obama: learning lip sync from audio
Suwajanakorn, S., Seitz, S. M., & Kemelmacher-Shlizerman, I. (2017) · ACM Transactions on Graphics · doi:10.1145/3072959.3073640 · Cited by 1,057
Capsule-forensics: Using Capsule Networks to Detect Forged Images and Videos
Nguyen, H. H., Yamagishi, J., & Echizen, I. (2019) · ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2019.8682602 · Cited by 951
Exploiting Visual Artifacts to Expose Deepfakes and Face Manipulations
Matern, F., Riess, C., & Stamminger, M. (2019) · 2019 IEEE Winter Applications of Computer Vision Workshops (WACVW) · doi:10.1109/wacvw.2019.00020 · Cited by 824 · cited 1×
“A few of the detection techniques targeted handcrafted features [24], [26], [41] like blinking inconsistencies, biological signals, and unrealistic details.”
BackgroundASVspoof 2019: Future Horizons in Spoofed and Fake Audio Detection
Todisco, M., Wang, X., Vestman, V., Sahidullah, M., Delgado, H., Nautsch, A., Yamagishi, J., Evans, N., Kinnunen, T., & Lee, K. A. (2019) · arXiv · doi:10.48550/arxiv.1904.05441 · Cited by 688 · cited 4×
“The ASVSpoof2015 challenge [76], ASVSpoof2017 challenge [29], and ASVSpoof2019 challenge [63] have helped spur deepfake audio research, referred to in the research community as spoof detection .”
BackgroundMethodResultInfluentialASVspoof 2019: Future Horizons in Spoofed and Fake Audio Detection
Todisco, M., Wang, X., Vestman, V., Sahidullah, M., Delgado, H., Nautsch, A., Yamagishi, J., Evans, N., Kinnunen, T. H., & Lee, K. A. (2019) · Interspeech 2019 · doi:10.21437/interspeech.2019-2249 · Cited by 687
Few-Shot Unsupervised Image-to-Image Translation
Liu, M.-Y., Huang, X., Mallya, A., Karras, T., Aila, T., Lehtinen, J., & Kautz, J. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.01065 · Cited by 646
Demo of Face2Face: real-time face capture and reenactment of RGB videos
Thies, J., Zollhöfer, M., Stamminger, M., Theobalt, C., & Nießner, M. (2016) · ACM SIGGRAPH 2016 Emerging Technologies · doi:10.1145/2929464.2929475 · Cited by 589
Invertible Conditional GANs for image editing
Perarnau, G., van de Weijer, J., Raducanu, B., & Álvarez, J. M. (2016) · arXiv · doi:10.48550/arxiv.1611.06355 · Cited by 563
Multi-task Learning For Detecting and Segmenting Manipulated Facial Images and Videos
Nguyen, H. H., Fang, F., Yamagishi, J., & Echizen, I. (2019) · arXiv · doi:10.48550/arxiv.1906.06876 · Cited by 551
Multi-task Learning for Detecting and Segmenting Manipulated Facial Images and Videos
Nguyen, H. H., Fang, F., Yamagishi, J., & Echizen, I. (2019) · 2019 IEEE 10th International Conference on Biometrics Theory, Applications and Systems (BTAS) · doi:10.1109/btas46853.2019.9185974 · Cited by 549 · cited 3×
“fake detection models proposed in prior work: CapsuleForensics [21], ClassNSeg [25], ConvLSTM [20], FaceNetLSTM [33], DenseNetAligned [47], and XceptionNet [32].”
MethodThe ASVspoof 2017 challenge: assessing the limits of replay spoofing attack detection
Kinnunen, T., Sahidullah, M., Delgado, H., Todisco, M., Evans, N., Yamagishi, J., & Lee, K. A. (2017) · Interspeech 2017 · doi:10.21437/interspeech.2017-1111 · Cited by 535 · cited 1×
“The ASVSpoof2015 challenge [76], ASVSpoof2017 challenge [29], and ASVSpoof2019 challenge [63] have helped spur deepfake audio research, referred to in the research com-”
BackgroundFast Face-Swap Using Convolutional Neural Networks
Korshunova, I., Shi, W., Dambre, J., & Theis, L. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.397 · Cited by 501
FakeCatcher: Detection of Synthetic Portrait Videos using Biological Signals
Ciftci, U. A., Demir, I., & Yin, L. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2020.3009287 · Cited by 499
DeepFakes: a New Threat to Face Recognition? Assessment and Detection
Korshunov, P., & Marcel, S. (2018) · arXiv · doi:10.48550/arxiv.1812.08685 · Cited by 497 · cited 1×
“DeepfakeTIMIT [71] was built from the VidTIMIT [72] dataset using a GAN-based face swap [5] over dataset subjects.”
MethodASVspoof 2015: the first automatic speaker verification spoofing and countermeasures challenge
Wu, Z., Kinnunen, T., Evans, N., Yamagishi, J., Hanilçi, C., Sahidullah, M., & Sizov, A. (2015) · Interspeech 2015 · doi:10.21437/interspeech.2015-462 · Cited by 484 · cited 1×
“The ASVSpoof2015 challenge [76], ASVSpoof2017 challenge [29], and ASVSpoof2019 challenge [63] have helped spur deepfake audio research, referred to in the research com-”
BackgroundKim, H., Garrido, P., Tewari, A., Xu, W., Thies, J., Niessner, M., Pérez, P., Richardt, C., Zollhöfer, M., & Theobalt, C. (2018) · ACM Transactions on Graphics · doi:10.1145/3197517.3201283 · Cited by 475 · cited 1×
“Kim et al. [39] accomplished a similar transfer, with the difference being that they transfer expressions as well as 3D pose into a target image, creating a video in the process.”
BackgroundRecurrent convolutional strategies for face manipulation detection in videos
Sabir, E., Cheng, J., Jaiswal, A., AbdAlmageed, W., Masi, I., & Natarajan, P. (2019) · arXiv · doi:10.48550/arxiv.1905.00582 · Cited by 443 · cited 5×
“Table V shows the test accuracies for our models as well as for the DenseNetAligned and XceptionNet models, two of the best models in our results for uncompressed video.”
BackgroundMethodInfluential(2019) · Cited by 415
The Deepfake Detection Challenge (DFDC) Preview Dataset
Dolhansky, B., Howes, R., Pflaum, B., Baram, N., & Ferrer, C. C. (2019) · arXiv · doi:10.48550/arxiv.1910.08854 · Cited by 413 · cited 2×
“Deepfake Detection Challenge (DFDC) [73] is a preview version of a larger dataset from a challenge developed by Facebook, and it consists of 5000 videos (original and manipulated).”
BackgroundMethodFew-shot Video-to-Video SynthesisPreprint
Wang, T.-C., Liu, M.-Y., Tao, A., Liu, G., Kautz, J., & Catanzaro, B. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.12713 · Cited by 401
“StarGAN-VC: non-parallel many-to-many voice conversion using star generative adversarial networks,”
Kameoka, H., Kaneko, T., Tanaka, K., & Hojo, N. (2018) · 2018 IEEE Spoken Language Technology Workshop (SLT) · doi:10.1109/slt.2018.8639535 · Cited by 371
A New Feature for Automatic Speaker Verification Anti-Spoofing: Constant Q Cepstral Coefficients
Todisco, M., Delgado, H., & Evans, N. (2016) · The Speaker and Language Recognition Workshop (Odyssey 2016) · doi:10.21437/odyssey.2016-41 · Cited by 335 · cited 5×
“Their best methods use a GMM Universal Background Model to fuse traditional features (MFCC, spectrogram, CQCCs, etc.) with an autoencoder neural network latent vector representation.”
MethodInfluentialThe Voice Conversion Challenge 2018: Promoting Development of Parallel and Nonparallel Methods
Lorenzo-Trueba, J., Yamagishi, J., Toda, T., Saito, D., Villavicencio, F., Kinnunen, T., & Ling, Z. (2018) · The Speaker and Language Recognition Workshop (Odyssey 2018) · doi:10.21437/odyssey.2018-28 · Cited by 305
Multi-Region Probabilistic Histograms for Robust and Scalable Identity Inference
Sanderson, C., & Lovell, B. C. (2009) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-01793-3_21 · Cited by 301
Attribute-Guided Face Generation Using Conditional CycleGAN
Lu, Y., Tai, Y.-W., & Tang, C.-K. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01258-8_18 · Cited by 233 · cited 1×
“Lu et al. [38] proposed an identity-guided conditional CycleGAN to translate low-resolution face images to high-resolution face images.”
BackgroundForensicTransfer: Weakly-supervised Domain Adaptation for Forgery Detection
Cozzolino, D., Thies, J., Rössler, A., Riess, C., Nießner, M., & Verdoliva, L. (2018) · arXiv · doi:10.48550/arxiv.1812.02510 · Cited by 221 · cited 2×
“Cozzolino et al. [23] applied an autoencoder-based architecture to show its usefulness for transfer learning.”
BackgroundMethodDeep Residual Neural Networks for Audio Spoofing Detection
Alzantot, M., Wang, Z., & Srivastava, M. B. (2019) · Interspeech 2019 · doi:10.21437/interspeech.2019-3174 · Cited by 215
Neural Voice Cloning with a Few Samples
Arık, S. Ö., Chen, J., Peng, K., Ping, W., & Zhou, Y. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.06006 · Cited by 172 · cited 1×
“Arik et al. [53] introduced a voice cloning model based on speaker adaptation and speaker encoding that .”
BackgroundIn Ictu Oculi: Exposing AI Generated Fake Face Videos by Detecting Eye Blinking
Li, Y., Chang, M., & Lyu, S. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1806.02877 · Cited by 169 · cited 2×
“Detection methods have concentrated on structure [18]–[23], [25] or soft biometrics [24], [26], [27] in visual data, as well as spoof detection for”
BackgroundMethodMarioNETte: Few-Shot Face Reenactment Preserving Identity of Unseen Targets
Ha, S., Kersner, M., Kim, B., Seo, S., & Kim, D. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i07.6721 · Cited by 157
Report: A Vector Quantization Approach to Speaker Recognition
Soong, F. K., Rosenberg, A. E., Juang, B.-H., & Rabiner, L. R. (1987) · AT&T Technical Journal · doi:10.1002/j.1538-7305.1987.tb00198.x · Cited by 148 · cited 1×
“Early speaker identification techniques used vector quantization [59], GMMs, and feature decomposition paired with Support Vector Machines (SVMs) [60].”
MethodFletcher, J. (2018) · Theatre Journal · doi:10.1353/tj.2018.0097 · Cited by 147 · cited 1×
“WHILE misinformation being spread through interpersonal communication and mass media is hardly new, we are at a pivotal point in history, where advances in digital communication and AI threaten to magnify the scale, persistence, and consequence of misinformation [1], [2].”
BackgroundCeleb-DF: A New Dataset for DeepFake ForensicsPreprint
Li, Y., Yang, X., Sun, P., Qi, H., & Lyu, S. (2019) · arXiv (Cornell University) · Cited by 137
A straightforward and efficient implementation of the factor analysis model for speaker verification
Matrouf, D., Scheffer, N., Fauve, B., & Bonastre, J.-F. (2007) · Interspeech (USB) · doi:10.21437/interspeech.2007-156 · Cited by 124 · cited 1×
“Early speaker identification techniques used vector quantization [59], GMMs, and feature decomposition paired with Support Vector Machines (SVMs) [60].”
MethodATTS2S-VC: Sequence-to-sequence Voice Conversion with Attention and Context Preservation Mechanisms
Tanaka, K., Kameoka, H., Kaneko, T., & Hojo, N. (2019) · ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2019.8683282 · Cited by 104
Ensemble Models for Spoofing Detection in Automatic Speaker Verification
Chettri, B., Stoller, D., Morfi, V., Ramírez, M. A. M., Benetos, E., & Sturm, B. L. (2019) · Interspeech 2019 · doi:10.21437/interspeech.2019-2505 · Cited by 103
End-to-end audio replay attack detection using deep convolutional networks with attention
Tom, F., Jain, M., & Dey, P. (2018) · Interspeech 2018 · doi:10.21437/interspeech.2018-2279 · Cited by 89
The Voice Conversion Challenge 2018: Promoting Development of Parallel and Nonparallel MethodsPreprint
Lorenzo-Trueba, J., Yamagishi, J., Toda, T., et al. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1804.04262 · Cited by 66 · cited 2×
“Deepfake creation methods typically rely on face swap-ping [3], [5] or face synthesis [6] combined with audio dubbing, voice conversion or voice synthesis [28].”
BackgroundMethodLong Range Acoustic and Deep Features Perspective on ASVspoof 2019
Das, R. K., Yang, J., & Li, H. (2019) · 2019 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU) · doi:10.1109/asru46091.2019.9003845 · Cited by 65 · cited 1×
“Das et al. [65] proposed to use long-range acoustic features based on the octave power spectrum instead of the linear power spectrum for spoofing detection.”
BackgroundDeferred Neural Rendering: Image Synthesis using Neural Textures
Thies, J., Zollhöfer, M., & Nießner, M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1904.12356 · Cited by 46
Poster: Towards Robust Open-World Detection of Deepfakes
Sohrawardi, S. J., Chintha, A., Thai, B., Seng, S., Hickerson, A., Ptucha, R., & Wright, M. (2019) · Proceedings of the 2019 ACM SIGSAC Conference on Computer and Communications Security · doi:10.1145/3319535.3363269 · Cited by 41 · cited 5×
“Our architecture for deepfake detection is inspired by the XceptionNet [32] architecture along with recurrent processing used in ConvLSTM [20] and FaceNetLSTM [33].”
MethodInfluentialSelf-citationAnalysing The Predictions Of a CNN-Based Replay Spoofing Detection System
Chettri, B., Mishra, S., Sturm, B. L., & Benetos, E. (2018) · 2018 IEEE Spoken Language Technology Workshop (SLT) · doi:10.1109/slt.2018.8639666 · Cited by 34
Realistic Human Action Recognition with Audio Context
Wu, Q., Wang, Z., Deng, F., & Feng, D. D. (2010) · International Conference on Digital Image Computing: Techniques and Applications (DICTA) · doi:10.1109/dicta.2010.57 · Cited by 17 · cited 1×
“While their performance on manipulated videos [20] from the HOHA dataset [49] was quite good, performance on the larger and more complex FaceForensics++ [22] dataset was not as effective.”
BackgroundA Cross-Database Study of Voice Presentation Attack Detection
Korshunov, P., & Marcel, S. (2019) · Advances in computer vision and pattern recognition · doi:10.1007/978-3-319-92627-8_16 · Cited by 15
Ovadya, A., & Whittlestone, J. (2019) · arXiv · doi:10.48550/arxiv.1907.11274 · Cited by 15 · cited 1×
“WHILE misinformation being spread through interpersonal communication and mass media is hardly new, we are at a pivotal point in history, where advances in digital communication and AI threaten to magnify the scale, persistence, and consequence of misinformation [1], [2].”
BackgroundTowards robust audio spoofing detection: a detailed comparison of traditional and learned featuresPreprint
Balamurali, B. T., Lin, K. W. E., Lui, S., Chen, J., & Herremans, D. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.12439 · Cited by 4 · cited 1×
“[61] reviewed traditional and newer machine learning spoof detection methods.”
Background(2000) · Medical Entomology and Zoology
Research works printed but not linked: 1
- 10.1109/iccv.2017.244 · DOI · Paper page
Cited by
Deepfake Detection: A Systematic Literature Review
Rana, M. S., Nobi, M. N., Murali, B., & Sung, A. H. (2022) · IEEE Access · doi:10.1109/access.2022.3154404 · Cited by 573
Masood, M., Nawaz, M., Malik, K. M., Javed, A., Irtaza, A., & Malik, H. (2023) · Applied Intelligence · doi:10.1007/s10489-022-03766-z · Cited by 543
“[118] XceptionNet CNN with bidirectional LSTM network Accuracy=97.”
MethodDeep learning for deepfakes creation and detection: A survey
Nguyen, T. T., Nguyen, Q. V. H., Nguyen, D. T., Nguyen, D. T., Huynh-The, T., Nahavandi, S., Nguyen, T. T., Pham, Q.-V., & Nguyen, C. M. (2022) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2022.103525 · Cited by 464
“The LRCN consists of a feature extractor based on CNN, a sequence learning based on long short term memory (LSTM), and a state prediction based on a fully connected layer to predict probability of eye open and close state.”
MethodInfluentialDeep Learning for Deepfakes Creation and Detection: A Survey
Nguyen, T. T., Nguyen, Q. V. H., Nguyen, D. T., Nguyen, D. T., Huynh-The, T., Nahavandi, S., Nguyen, T. T., Pham, Q.-V., & Nguyen, C. M. (2019) · arXiv · doi:10.48550/arxiv.1909.11573 · Cited by 464
“The LRCN consists of a feature extractor based on CNN, a sequence learning based on long short term memory (LSTM), and a state prediction based on a fully connected layer to predict probability of eye open and close state.”
MethodInfluentialDeepfake detection using deep learning methods: A systematic and comprehensive review
Heidari, A., Jafari Navimipour, N., Dag, H., & Unal, M. (2024) · WIREs Data Mining and Knowledge Discovery · doi:10.1002/widm.1520 · Cited by 349
“Also, Chintha et al. (2020) developed the XcepTemporal convolutional RNN system for deepfake detection.”
Method
Metadata
- Authors
- Chintha, A., Thai, B., Sohrawardi, S. J., Bhatt, K., Hickerson, A., Wright, M., & Ptucha, R.
- Affiliations
- Rochester Institute of Technology (US)
- University of South Carolina (US)
- ORCID
- Venue
- IEEE Journal of Selected Topics in Signal Processing
- Published
- 2020-08
- Volume
- 14
- Issue
- 5
- Pages
- 1024-1037
- ISSN
- 1932-4553 (ISSN-L)
- 1941-0484
- OpenAlex
- W3033711348
- Cited by
- 226
- References
- 77
- Citations by year
- 2026: 36 · 2025: 40 · 2024: 51 · 2023: 49 · 2022: 30 · 2021: 17 · 2020: 3
- FWCI
- 9.62
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- Locations
- IEEE Journal of Selected Topics in Signal Processing (publishedVersion)
- Funders
- The Ethics and Governance of Artificial Intelligence Initiative (006409-002)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Digital Media Forensic Detection
- Physical Sciences › Computer Science › Artificial Intelligence › Speech Recognition and Synthesis
- Physical Sciences › Computer Science › Signal Processing › Biometric Identification and Security
- Keywords
- deepfake detection, recurrent convolutional networks, latent representations, generative adversarial networks, digital forensics, FaceForensics++, Celeb-DF, ASVspoof 2019
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.