A Deep Learning Approach to Universal Image Manipulation Detection Using a New Convolutional Layer
The DOI matches its Crossref record; 24 of the 24 references it cites were found in a registry or an index.
Abstract
When creating a forgery, a forger can modify an image using many different image editing operations. Since a forensic examiner must test for each of these, significant interest has arisen in the development of universal forensic algorithms capable of detecting many different image editing operations and manipulations. In this paper, we propose a universal forensic approach to performing manipulation detection using deep learning. Specifically, we propose a new convolutional network architecture capable of automatically learning manipulation detection features directly from training data. In their current form, convolutional neural networks will learn features that capture an image's content as opposed to manipulation detection features. To overcome this issue, we develop a new form of convolutional layer that is specifically designed to suppress an image's content and adaptively learn manipulation detection features. Through a series of experiments, we demonstrate that our proposed approach can automatically learn how to detect multiple image manipulations without relying on pre-selected features or any preprocessing. The results of these experiments show that our proposed approach can automatically detect several different manipulations with an average accuracy of 99.10%.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“It is threatening to world se- curity when deepfake methods can be employed to cre- ate videos of world leaders with fake speeches for falsi- fication purposes [10–12].”
From Deep learning for deepfakes creation and detection: A survey (2022) · cites this paper as [10]
Citations
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
Going deeper with convolutions
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298594 · Cited by 48,562
(2010) · International Conference on Machine Learning · Cited by 13,995
Advances in Neural Information Processing Systems 25
Zhang, Y., Sutton, C. A., Storkey, A., & Zoubin Ghahramani, A. (2012) · Cited by 12,653
Backpropagation Applied to Handwritten Zip Code Recognition
LeCun, Y., Boser, B., Denker, J. S., Henderson, D., Howard, R. E., Hubbard, W., & Jackel, L. D. (1989) · Neural Computation · doi:10.1162/neco.1989.1.4.541 · Cited by 11,891
Caffe: Convolutional Architecture for Fast Feature Embedding
Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., Guadarrama, S., & Darrell, T. (2014) · Proceedings of the 22nd ACM international conference on Multimedia · doi:10.1145/2647868.2654889 · Cited by 11,077
Improving neural networks by preventing co-adaptation of feature detectors
Hinton, G. E., Srivastava, N., Krizhevsky, A., Sutskever, I., & Salakhutdinov, R. R. (2012) · arXiv · doi:10.48550/arxiv.1207.0580 · Cited by 6,382
Convolutional networks for images, speech, and time series
LeCun, Y., & Bengio, Y. (1998) · HAL (Le Centre pour la Communication Scientifique Directe) · doi:10.5555/303568.303704 · Cited by 4,231
(1989) · neural information processing systems · Cited by 3,736
Rich Models for Steganalysis of Digital Images
Fridrich, J., & Kodovsky, J. (2012) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2012.2190402 · Cited by 2,144
LeCun, Y. A., Bottou, L., Orr, G. B., & Müller, K.-R. (2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-35289-8_3 · Cited by 1,264
Steganalysis by Subtractive Pixel Adjacency Matrix
Pevny, T., Bas, P., & Fridrich, J. (2010) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2010.2045842 · Cited by 887
Steganalysis by subtractive pixel adjacency matrix
Pevný, T., Bas, P., & Fridrich, J. (2009) · Proceedings of the 11th ACM workshop on Multimedia and security · doi:10.1145/1597817.1597831 · Cited by 769
Exposing digital forgeries by detecting traces of resampling
Popescu, A., & Farid, H. (2005) · doi:10.1109/tsp.2004.839932(410 · Cited by 655
Median Filtering Forensics Based on Convolutional Neural Networks
Chen, J., Kang, X., Liu, Y., & Wang, Z. J. (2015) · IEEE Signal Processing Letters · doi:10.1109/lsp.2015.2438008 · Cited by 425
Information Forensics: An Overview of the First Decade
Stamm, M. C., Wu, M., & Liu, K. J. R. (2013) · IEEE Access · doi:10.1109/access.2013.2260814 · Cited by 394
Self-citationForensic detection of image manipulation using statistical intrinsic fingerprints
Stamm, M. C., & Liu, K. J. R. (2010) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2010.2053202 · Cited by 321
Self-citationOn detection of median filtering in digital images
Kirchner, M., & Fridrich, J. (2010) · SPIE Proceedings · doi:10.1117/12.839100 · Cited by 278
SPIE (2013) · SPIE Professional · doi:10.1117/2.4201304.17 · Cited by 240
Fast and reliable resampling detection by spectral analysis of fixed linear predictor residue
Kirchner, M. (2008) · Proceedings of the 10th ACM workshop on Multimedia and security · doi:10.1145/1411328.1411333 · Cited by 231
Robust Median Filtering Forensics Using an Autoregressive Model
Kang, X., Stamm, M. C., Peng, A., & Liu, K. J. R. (2013) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2013.2273394 · Cited by 229
Self-citationA universal image forensic strategy based on steganalytic model
Qiu, X., Li, H., Luo, W., & Huang, J. (2014) · ACM Workshop on Information Hiding and Multimedia Security (IH&MMSec) · doi:10.1145/2600918.2600941 · Cited by 97
Camera model identification framework using an ensemble of demosaicing features
Chen, C., & Stamm, M. C. (2015) · IEEE International Workshop on Information Forensics and Security · doi:10.1109/wifs.2015.7368573 · Cited by 84
Self-citationGeneral-purpose image forensics using patch likelihood under image statistical models
Fan, W., Wang, K., & Cayre, F. (2015) · IEEE International Workshop on Information Forensics and Security · doi:10.1109/wifs.2015.7368606 · Cited by 66
Research works printed but not linked: 2
- Le Cun B. B., 1990, Citeseer
- Nair V., 2010, International Conference on Machine Learning, 807
Cited by
FaceForensics++: Learning to Detect Manipulated Facial Images
Rossler, A., Cozzolino, D., Verdoliva, L., Riess, C., Thies, J., & Niessner, M. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00009 · Cited by 3,314
MesoNet: a Compact Facial Video Forgery Detection Network
Afchar, D., Nozick, V., Yamagishi, J., & Echizen, I. (2018) · 2018 IEEE International Workshop on Information Forensics and Security (WIFS) · doi:10.1109/wifs.2018.8630761 · Cited by 1,872
Deepfakes and beyond: A Survey of face manipulation and fake detection
Tolosana, R., Vera-Rodriguez, R., Fierrez, J., Morales, A., & Ortega-Garcia, J. (2020) · Information Fusion · doi:10.1016/j.inffus.2020.06.014 · Cited by 1,237
Face X-Ray for More General Face Forgery Detection
Li, L., Bao, J., Zhang, T., Yang, H., Chen, D., Wen, F., & Guo, B. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00505 · Cited by 1,211
Thinking in Frequency: Face Forgery Detection by Mining Frequency-Aware Clues
Qian, Y., Yin, G., Sheng, L., Chen, Z., & Shao, J. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58610-2_6 · Cited by 1,058
Metadata
- Authors
- Bayar, B., & Stamm, M. C.
- Affiliations
- Drexel University (US)
- Venue
- Proceedings of the 4th ACM Workshop on Information Hiding and Multimedia Security
- Publisher
- ACM
- Published
- 2016-06-20
- Pages
- 5-10
- OpenAlex
- W2412509443
- Cited by
- 838
- References
- 24
- Citations by year
- 2026: 43 · 2025: 94 · 2024: 91 · 2023: 117 · 2022: 136 · 2021: 93 · 2020: 106 · 2019: 56 · 2018: 66 · 2017: 31 · 2016: 5
- FWCI
- 18.27
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- ACM Workshop on Information Hiding and Multimedia Security (IH&MMSec) (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Digital Media Forensic Detection
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Steganography and Watermarking Techniques
- Physical Sciences › Mathematics › Statistics and Probability › Benford’s Law and Fraud Detection
- Keywords
- image forgery detection, image forensics, convolutional neural networks, image tampering, deep learning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.