The DOI matches its Crossref record; 30 of the 31 references it cites were found in a registry or an index.
MFC Datasets: Large-Scale Benchmark Datasets for Media Forensic Challenge Evaluation
Abstract
We provide a benchmark for digital Media Forensics Challenge (MFC) evaluations. Our comprehensive data comprises over 176,000 high provenance (HP) images and 11,000 HP videos; more than 100,000 manipulated images and 4,000 manipulated videos; 35 million internet images and 300,000 video clips. We have designed and generated a series of development, evaluation, and challenge datasets, and used them to assess the progress and thoroughly analyze the performance of diverse systems on a variety of media forensics tasks in the past two years. In this paper, we first introduce the objectives, challenges, and approaches to building media forensics evaluation datasets. We then discuss our approaches to forensic dataset collection, annotation, and manipulation, and present the design and infrastructure to effectively and efficiently build the evaluation datasets to support various evaluation tasks. Given a specified query, we build an infrastructure that selects the customized evaluation subsets for the targeted analysis report. Finally, we demonstrate the evaluation results in the past evaluations.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Test results show that the popular face recog- nition systems based on VGG [65] and Facenet [59, 66] are unable to detect deepfakes effectively.”
From Deep learning for deepfakes creation and detection: A survey (2022) · cites this paper as [59]
Citations
Cites
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
Microsoft COCO: Common Objects in Context
Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., & Zitnick, C. L. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10602-1_48 · Cited by 45,460
Vision meets robotics: The KITTI dataset
Geiger, A., Lenz, P., Stiller, C., & Urtasun, R. (2013) · The International Journal of Robotics Research · doi:10.1177/0278364913491297 · Cited by 9,913
The Pascal Visual Object Classes Challenge: A Retrospective
Everingham, M., Eslami, S. M. A., Van Gool, L., Williams, C. K. I., Winn, J., & Zisserman, A. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-014-0733-5 · Cited by 7,139
Progressive Growing of GANs for Improved Quality, Stability, and Variation
Karras, T., Aila, T., Laine, S., & Lehtinen, J. (2017) · arXiv · doi:10.48550/arxiv.1710.10196 · Cited by 4,727
SUN database: Large-scale scene recognition from abbey to zoo
Xiao, J., Hays, J., Ehinger, K. A., Oliva, A., & Torralba, A. (2010) · 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2010.5539970 · Cited by 3,283
80 Million Tiny Images: A Large Data Set for Nonparametric Object and Scene Recognition
Torralba, A., Fergus, R., & Freeman, W. T. (2008) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2008.128 · Cited by 1,921
Thomée, B., Shamma, D. A., Friedland, G. H., Elizalde, B., Ni, K. S., Poland, D., Borth, D., & Li, L.-J. (2016) · Communications of the ACM · doi:10.1145/2812802 · Cited by 1,715
Evaluation campaigns and TRECVid
Smeaton, A. F., Over, P., & Kraaij, W. (2006) · doi:10.1145/1178677.1178722 · Cited by 1,227
UCID: an uncompressed color image database
Schaefer, G., & Stich, M. (2003) · Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE · doi:10.1117/12.525375 · Cited by 1,078
Research works printed but not linked: 17
- rossler, 2018, FaceForensics A Large-scale Video Dataset for Forgery Detection in Human Faces
- dong, 2013, CASIA image tampering detection evaluation database, in Signal and Information Processing (ChinaSIP), IEEE China Summit & International Conference on, 422
- 2015, A Database of Photos of Plants Animals Habitats and Other Natural History Subjects
- bansal, 2016, Umdfaces An annotated face dataset for training deep networks
- bas, 2011, Break Our Steganographic System, The Ins and Outs of Organizing BOSS INFORMATION HIDING Czech Republic 6958/2011 59-70 Lecture Notes in Computer Science
- 0, DARPA MediFor
- yates, 2017, Nimble Challenge 2017 Evaluation Plan, NIST
- yates, 2018, Media Forensics Challenge 2018 Evaluation Plan, NIST
- griffin, 2007, Caltech-256 Object Category Dataset California Institute of Technology
- 0, EU project REWIND (REVerse engineering of audio-VIsual coNtent Data)
- rocha, 0, The First IFS-TC Image Forensics Challenge
- boehme, 2012, Counter-Forensics Attacking Image Forensics Digital Image Forensics, 327
- wang, 2018, Video-to-Video Synthesis
- lyu, 2012, Natural Image Statistics in Digital Image Forensics, Digital Image Forensics, 239
- boylan, 2018, Will Deep-Fake Technology Destroy Democracy?
- robertson, 2018, Manual for MaskGen Journaling Tool
- schellmann, 2018, Deepfake Videos Are Getting Real and That's a Problem
Cited by
Sorted by citations
FaceForensics++: Learning to Detect Manipulated Facial Images
Rossler, A., Cozzolino, D., Verdoliva, L., Riess, C., Thies, J., & Niessner, M. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00009 · Cited by 3,314
Deepfakes and beyond: A Survey of face manipulation and fake detection
Tolosana, R., Vera-Rodriguez, R., Fierrez, J., Morales, A., & Ortega-Garcia, J. (2020) · Information Fusion · doi:10.1016/j.inffus.2020.06.014 · Cited by 1,237
Exposing Deep Fakes Using Inconsistent Head Poses
Yang, X., Li, Y., & Lyu, S. (2019) · ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2019.8683164 · Cited by 1,196
Media Forensics and DeepFakes: An Overview
Verdoliva, L. (2020) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2020.3002101 · Cited by 979
Deep learning for deepfakes creation and detection: A survey
Nguyen, T. T., Nguyen, Q. V. H., Nguyen, D. T., Nguyen, D. T., Huynh-The, T., Nahavandi, S., Nguyen, T. T., Pham, Q.-V., & Nguyen, C. M. (2022) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2022.103525 · Cited by 464
“real images and 252 deep fake images, which is a subset of data used in the DARPA MediFor GAN Image/Video Challenge [129].”
Method
Metadata
- Authors
- Guan, H., Kozak, M., Robertson, E., Lee, Y., Yates, A. N., Delgado, A., Zhou, D., Kheyrkhah, T., Smith, J., & Fiscus, J.
- Affiliations
- National Institute of Standards and Technology (US)
- Thermo Fisher Scientific (Israel) (IL)
- Venue
- 2019 IEEE Winter Applications of Computer Vision Workshops (WACVW)
- Publisher
- IEEE
- Published
- 2019-01
- Pages
- 63-72
- OpenAlex
- W2912308702
- Cited by
- 329
- References
- 31
- Citations by year
- 2026: 48 · 2025: 71 · 2024: 73 · 2023: 48 · 2022: 35 · 2021: 25 · 2020: 20 · 2019: 8
- FWCI
- 7.17
- Type
- conference paper
- Language
- en
- Open access
- closed
- Licence
- Locations
- IEEE/CVF Winter Conference on Applications of Computer Vision Workshops (WACVW) (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Digital Media Forensic Detection
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Steganography and Watermarking Techniques
- Physical Sciences › Mathematics › Statistics and Probability › Benford’s Law and Fraud Detection
- Keywords
- multimedia forensics, benchmark dataset, manipulated images, manipulated video, image forgery detection, video manipulation detection, data annotation, dataset collection
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
