Speaker Inconsistency Detection in Tampered Video
Open accessAbstract
With the increasing amount of video being consumed by people daily, there is a danger of the rise in maliciously modified video content (i.e., `fake news') that could be used to damage innocent people or to impose a certain agenda, e.g., meddle in elections. In this paper, we consider audio manipulations in video of a person speaking to the camera. Such manipulation is easy to perform, for instance, one can just replace a part of audio, while it can dramatically change the message and the meaning of the video. With the goal to develop an automated system that can detect these audio-visual speaker inconsistencies, we consider several approaches proposed for lip-syncing and dubbing detection, based on convolutional and recurrent networks and compare them with systems that are based on more traditional classifiers. We evaluated these methods on publicly available databases VidTIMIT, AMI, and GRID, for which we generated sets of tampered data.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
The citation lists haven’t been fetched yet
References 14 · Cited by 97
Research works printed but not linked: 1
- [4]4. J. S. Chung, A. W. Senior, O. Vinyals, and A. Zisserman, "Lip reading sentences in the wild," CoRR, vol. abs/1611.05358, 2016. [Online]. Available: http://arxiv.org/abs/1611.05358 · arXiv
- Authors
- Korshunov, P., & Marcel, S.
- Affiliations
- Idiap Research Institute (CH)
- ORCID
- Korshunov, Pavel
- Venue
- 2018 26th European Signal Processing Conference (EUSIPCO)
- Publisher
- IEEE
- Published
- 2018-09
- Pages
- 2375-2379
- OpenAlex
- W2903369540
- Cited by
- 97
- References
- 14
- Citations by year
- 2026: 4 · 2025: 9 · 2024: 18 · 2023: 16 · 2022: 18 · 2021: 15 · 2020: 10 · 2019: 6 · 2018: 1
- FWCI
- 2.41
- Type
- conference paper
- Language
- en
- Open access
- green
- iris.unil.ch
- Locations
- European Signal Processing Conference (EUSIPCO) (publishedVersion)
- IRIS (submittedVersion)
- Infoscience (Ecole Polytechnique Fédérale de Lausanne) (submittedVersion, cc-by-nc-nd)
- Funders
- Defense Advanced Research Projects Agency (FA8750)
- Advanced Research Projects Agency
- Air Force Research Laboratory
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Digital Media Forensic Detection
- Physical Sciences › Computer Science › Signal Processing › Speech and Audio Processing
- Physical Sciences › Computer Science › Artificial Intelligence › Speech Recognition and Synthesis
- Keywords
- sound manipulation, fake news, video forensics, speaker verification, AMI, grids
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
