Detecting Deep-Fake Videos from Aural and Oral Dynamics
The DOI matches its Crossref record; 13 of the 13 references it cites were found in a registry or an index.
Abstract
A face-swap deep fake replaces a person’s face – from eyebrows to chin – with another face. A lip-sync deep fake replaces a person’s mouth region to be consistent with an impersonated or synthesized audio track. An overlooked aspect in the creation of these deep-fake videos is the human ear. Statically, the shape of the human ear has been shown to provide a biometric signal. Dynamically, movement of the mandible (lower jaw) causes changes in the shape of the ear and ear canal. While the facial identity in a face-swap deep fake may accurately depict the co-opted identity, the ears belong to the original identity. While the mouth in a lip-sync deep fake may be well synchronized with the audio, the dynamics of the ear motion will be de-coupled from the mouth and jaw motion. We describe a forensic technique that exploits these static and dynamic aural properties.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Towards Evaluating the Robustness of Neural Networks
Carlini, N., & Wagner, D. (2017) · 2017 IEEE Symposium on Security and Privacy (SP) · doi:10.1109/sp.2017.49 · Cited by 8,859
Face X-Ray for More General Face Forgery Detection
Li, L., Bao, J., Zhang, T., Yang, H., Chen, D., Wen, F., & Guo, B. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00505 · Cited by 1,211
CNN-Generated Images Are Surprisingly Easy to Spot… for Now
Wang, S.-Y., Wang, O., Zhang, R., Owens, A., & Efros, A. A. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00872 · Cited by 1,210
Exposing Deep Fakes Using Inconsistent Head Poses
Yang, X., Li, Y., & Lyu, S. (2019) · ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2019.8683164 · Cited by 1,196
In Ictu Oculi: Exposing AI Created Fake Videos by Detecting Eye Blinking
Li, Y., Chang, M.-C., & Lyu, S. (2018) · 2018 IEEE International Workshop on Information Forensics and Security (WIFS) · doi:10.1109/wifs.2018.8630787 · Cited by 1,089
Exposing DeepFake Videos By Detecting Face Warping Artifacts
Li, Y., & Lyu, S. (2018) · arXiv · doi:10.48550/arxiv.1811.00656 · Cited by 785 · cited 1×
“Low-level techniques detect pixel-level, synthesis artifacts, including generic artifacts [41, 37, 40, 35], warping artifacts [26], and blending artifacts [24].”
BackgroundTwo-Stream Neural Networks for Tampered Face Detection
Zhou, P., Han, X., Morariu, V. I., & Davis, L. S. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2017.229 · Cited by 700
Video Rewrite: driving visual speech with audio
Bregler, C., Covell, M., & Slaney, M. (1997) · Proceedings of the 24th annual conference on Computer graphics and interactive techniques - SIGGRAPH '97 · doi:10.1145/258734.258880 · Cited by 669
Detecting and Simulating Artifacts in GAN Fake Images
Zhang, X., Karaman, S., & Chang, S.-F. (2019) · 2019 IEEE International Workshop on Information Forensics and Security (WIFS) · doi:10.1109/wifs47025.2019.9035107 · Cited by 567 · cited 1×
“Low-level techniques detect pixel-level, synthesis artifacts, including generic artifacts [41, 37, 40, 35], warping artifacts [26], and blending artifacts [24].”
BackgroundFakeCatcher: Detection of Synthetic Portrait Videos using Biological Signals
Ciftci, U. A., Demir, I., & Yin, L. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2020.3009287 · Cited by 499
Research works printed but not linked: 29
- yu, 2021, Responsible disclosure of generative models using scalable fingerprinting, arXiv 2012 08726
- yu, 2020, Artificial GAN fingerprints: Rooting deepfake attribution in training data, arXiv 2007 08457
- 10.1109/35.940053 · DOI · Paper page
- 10.1109/ICPR.2002.1044746 · DOI · Paper page
- tomasi, 1991, Technical Report CMU-CS-91-132
- 10.1111/j.1600-0404.1963.tb05317.x · DOI · Paper page
- yu, 2018, Attributing fake images to GANs: Learning and analyzing GAN fingerprints, IEEE International Conference on Computer Vision
- 10.1068/p060313 · DOI · Paper page
- 10.1109/ICPR.2000.906202 · DOI · Paper page
- 10.1109/FG.2018.00065 · DOI
- 10.1007/978-3-540-75759-7_97 · DOI
- 0, deeptomcruise
- emerši?, 2017, Ear recognition: More than a survey, Neurocomputing, 255, 26, 10.1016/j.neucom.2016.08.139 · DOI · Paper page
- 10.7551/mitpress/10451.001.0001 · DOI · Paper page
- 10.1121/1.1430682 · doi:10.1121/1.1430682
- 10.25080/Majora-7b98e3ed-003 · DOI · Paper page
- 10.1145/2431211.2431221 · DOI · Paper page
- agarwal, 2019, Protecting world leaders against deep fakes, Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, 38
- 10.1007/s10489-020-01995-8 · DOI
- agarwal, 2020, Detecting deep-fake videos from phoneme-viseme mismatches, Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, 660
- baltrušaitis, 2016, OpenFace: an open source facial behavior analysis toolkit, IEEE Winter Conference on Applications of Computer Vision, 1
- 10.1109/BTAS.2010.5634535 · DOI
- burge, 1996, Ear biometrics, Biometrics, 273
- 0, Content Authenticity Initiative
- hartley, 2003, Multiple View Geometry in Computer Vision
- 10.2466/pms.1978.47.3.857 · DOI · Paper page
- 10.3758/BF03195300 · DOI · Paper page
- 10.1109/FG.2017.79 · DOI
- king, 2009, Dlib-ML: A machine learning toolkit, Journal of Machine Learning Research
Cited by
Countering Malicious DeepFakes: Survey, Battleground, and Horizon
Juefei-Xu, F., Wang, R., Huang, Y., Guo, Q., Ma, L., & Liu, Y. (2022) · International Journal of Computer Vision · doi:10.1007/s11263-022-01606-8 · Cited by 191
Dynamic Difference Learning With Spatio–Temporal Correlation for Deepfake Video Detection
Yin, Q., Lu, W., Li, B., & Huang, J. (2023) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2023.3290752 · Cited by 92
Delving into the Local: Dynamic Inconsistency Learning for DeepFake Video Detection
Gu, Z., Chen, Y., Yao, T., Ding, S., Li, J., & Ma, L. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i1.19955 · Cited by 93
“(Agarwal and Farid 2021) describes a forensic technique that exploits the abnormal shape of the ear and ear canal caused by movement of lower jaw.”
BackgroundGamage, D., Ghasiya, P., Bonagiri, V., Whiting, M. E., & Sasahara, K. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3517446 · Cited by 79
“deepfake techniques [9, 13, 55, 57] and methods for detecting deepfakes [1, 33, 34, 46, 67].”
MethodHuman detection of political speech deepfakes across transcripts, audio, and video
Groh, M., Sankaranarayanan, A., Singh, N., Kim, D. Y., Lippman, A., & Picard, R. (2024) · Nature Communications · doi:10.1038/s41467-024-51998-z · Cited by 89
Metadata
- Authors
- Agarwal, S., & Farid, H.
- Venue
- 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)
- Publisher
- IEEE
- Published
- 2021-06
- Pages
- 981-989
- OpenAlex
- W3174572554
- Cited by
- 66
- References
- 13
- FWCI
- 3.81
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Detecting Deep-Fake Videos from Aural and Oral Dynamics. https://app.cochat.ai/papers/10.1109/cvprw53098.2021.00109
- The paper
- Agarwal, S., & Farid, H. (2021). Detecting Deep-Fake Videos from Aural and Oral Dynamics. 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). https://doi.org/10.1109/cvprw53098.2021.00109
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.