Psychophysical Evaluation of Human Performance in Detecting Digital Face Image Manipulations
Open accessThe DOI matches its Crossref record; 70 of the 70 references it cites were found in a registry or an index.
Abstract
In recent years, increasing deployment of face recognition technology in security-critical settings, such as border control or law enforcement, has led to considerable interest in the vulnerability of face recognition systems to attacks utilising legitimate documents, which are issued on the basis of digitally manipulated face images. As automated manipulation and attack detection remains a challenging task, conventional processes with human inspectors performing identity verification remain indispensable. These circumstances merit a closer investigation of human capabilities in detecting manipulated face images, as previous work in this field is sparse and often concentrated only on specific scenarios and biometric characteristics. This work introduces a web-based, remote visual discrimination experiment on the basis of principles adopted from the field of psychophysics and subsequently discusses interdisciplinary opportunities with the aim of examining human proficiency in detecting different types of digitally manipulated face images, specifically face swapping, morphing, and retouching. In addition to analysing appropriate performance measures, a possible metric of detectability is explored. Experimental data of 306 probands indicate that detection performance is widely distributed across the population and detection of certain types of face image manipulations is much more challenging than others.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Overall, deepfake detection does not seem to vary based on gender, education, income, or profession [7, 32, 37].”
From Human Performance in Deepfake Detection: A Systematic Review (2025) · cites this paper as [7]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Type of stimuli (video versus audio) and the technique used for deepfake generation (e.g., face swapping, lip-syncing, and level of morphing) demonstrated mixed results [7, 20, 22, 28, 30, 42].”
From Human Performance in Deepfake Detection: A Systematic Review (2025) · cites this paper as [7]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Type of stimuli (video versus audio) and the technique used for deepfake generation (e.g., face swapping, lip-syncing, and level of morphing) demonstrated mixed results [7, 20, 22, 28, 30, 42].”
From Human Performance in Deepfake Detection: A Systematic Review (2025) · cites this paper as [7]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Furthermore, Nichols et al. found that participants were more likely to make false-negative errors (i.e., saying a photo is fake, when it is real) for lower-quality morphs. The inverse was true for higher-quality morphs -participants were more likely to make false-positive errors (i.e., saying a photo is real, when it is fake).”
From Human Performance in Deepfake Detection: A Systematic Review (2025) · cites this paper as [7]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Almost all video/image stimuli used face swapping or lip-sync technique, except for three studies that used morphing techniques [6, 7, 18].”
From Human Performance in Deepfake Detection: A Systematic Review (2025) · cites this paper as [7]
5 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
ArcFace: Additive Angular Margin Loss for Deep Face Recognition
Deng, J., Guo, J., Xue, N., & Zafeiriou, S. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00482 · Cited by 7,781 · cited 1×
“Similarity scores were computed for each manipulated face image from a comparison with the bona fide source image, by means of ArcFace face recognition system (FRS), based on a deep convolutional neural network (DCNN) approach utilizing Additive Angular Margin Loss [80], for”
MethodSensory Evaluation of Food: Principles and Practices
Lawless, H. T., & Heymann, H. (2010) · Food Science Text Series · doi:10.1007/978-1-4419-6488-5 · Cited by 4,815
FaceForensics++: Learning to Detect Manipulated Facial Images
Rossler, A., Cozzolino, D., Verdoliva, L., Riess, C., Thies, J., & Niessner, M. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00009 · Cited by 3,314 · cited 5×
“One of few studies examining alternative manipulation methods is [54], where the human ability to distinguish deepfake (i.e. face swapping) images from originals was assessed.”
BackgroundMethodResultInfluentialCalculation of signal detection theory measures
Stanislaw, H., & Todorov, N. (1999) · Behavior Research Methods, Instruments, & Computers · doi:10.3758/bf03207704 · Cited by 2,673
Overview of the Face Recognition Grand Challenge
Phillips, P. J., Flynn, P. J., Scruggs, T., Bowyer, K. W., Chang, J., Hoffman, K., Marques, J., Min, J., & Worek, W. (2005) · 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) · doi:10.1109/cvpr.2005.268 · Cited by 2,438
The FERET database and evaluation procedure for face-recognition algorithms
Phillips, P. J., Wechsler, H., Huang, J., & Rauss, P. J. (1998) · Image and Vision Computing · doi:10.1016/s0262-8856(97)00070-x · Cited by 2,431
The psychometric function: I. Fitting, sampling, and goodness of fit
Wichmann, F. A., & Hill, N. J. (2001) · Perception & Psychophysics · doi:10.3758/bf03194544 · Cited by 2,358
First Impressions: Making Up Your Mind After a 100-Ms Exposure to a Face
Willis, J., & Todorov, A. (2006) · Psychological Science · doi:10.1111/j.1467-9280.2006.01750.x · Cited by 2,300
J., J., & Fechner, G. T. (1889) · The American Journal of Psychology · doi:10.2307/1411906 · Cited by 1,429
Macmillan, N. A., & Creelman, C. D. (2004) · doi:10.4324/9781410611147 · Cited by 1,269
Deepfakes and beyond: A Survey of face manipulation and fake detection
Tolosana, R., Vera-Rodriguez, R., Fierrez, J., Morales, A., & Ortega-Garcia, J. (2020) · Information Fusion · doi:10.1016/j.inffus.2020.06.014 · Cited by 1,237
Duchaine, B., & Nakayama, K. (2006) · Neuropsychologia · doi:10.1016/j.neuropsychologia.2005.07.001 · Cited by 1,036
Farid, H. (2009) · IEEE Signal Processing Magazine · doi:10.1109/msp.2008.931079 · Cited by 925
Feature-based image metamorphosis
Beier, T., & Neely, S. (1992) · Proceedings of the 19th annual conference on Computer graphics and interactive techniques · doi:10.1145/133994.134003 · Cited by 914
Corrections for extreme proportions and their biasing effects on estimated values ofd′
Hautus, M. J. (1995) · Behavior Research Methods, Instruments, & Computers · doi:10.3758/bf03203619 · Cited by 894
The Glasgow Face Matching Test
Burton, A. M., White, D., & McNeill, A. (2010) · Behavior Research Methods · doi:10.3758/brm.42.1.286 · Cited by 472
Modern Techniques in Neuroscience Research
(1999) · Springer eBooks · doi:10.1007/978-3-642-58552-4 · Cited by 452
SimSwap: An Efficient Framework For High Fidelity Face Swapping
Chen, R., Chen, X., Ni, B., & Ge, Y. (2020) · Proceedings of the 28th ACM International Conference on Multimedia · doi:10.1145/3394171.3413630 · Cited by 418 · cited 1×
“features of the target image towards identity information from source image features [25].”
BackgroundSchütt, H. H., Harmeling, S., Macke, J. H., & Wichmann, F. A. (2016) · Vision Research · doi:10.1016/j.visres.2016.02.002 · Cited by 402
Forensic detection of image manipulation using statistical intrinsic fingerprints
Stamm, M. C., & Liu, K. J. R. (2010) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2010.2053202 · Cited by 321
Ferrara, M., Franco, A., & Maltoni, D. (2014) · IEEE International Joint Conference on Biometrics · doi:10.1109/btas.2014.6996240 · Cited by 278
Passport officers’ errors in face matching
White, D., Kemp, R. I., Jenkins, R., Matheson, M., & Burton, A. M. (2014) · PLoS ONE · doi:10.1371/journal.pone.0103510 · Cited by 241
MIPGAN—Generating Strong and High Quality Morphing Attacks Using Identity Prior Driven GAN
Zhang, H., Venkatesh, S. K., Ramachandra, R., Raja, K. B., Damer, N., & Busch, C. (2021) · IEEE Transactions on Biometrics Behavior and Identity Science · doi:10.1109/tbiom.2021.3072349 · Cited by 182
Self-citationEhrenstein, W. H., & Ehrenstein, A. (1999) · Modern Techniques in Neuroscience Research · doi:10.1007/978-3-642-58552-4_43 · Cited by 178
The Gale encyclopedia of psychology
(1997) · Choice Reviews Online · doi:10.5860/choice.34-4198 · Cited by 145
Visual Mis- and Disinformation, Social Media, and Democracy
Dan, V., Paris, B., Donovan, J., Hameleers, M., Roozenbeek, J., van der Linden, S., & von Sikorski, C. (2021) · Journalism & Mass Communication Quarterly · doi:10.1177/10776990211035395 · Cited by 144 · cited 1×
“societal concerns, such as the spread of misinformation via social media [14].”
Detecting Photoshopped Faces by Scripting Photoshop
Wang, S.-Y., Wang, O., Zhang, R., Owens, A., & Efros, A. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.01017 · Cited by 135
Damer, N., Saladie, A. M., Braun, A., & Kuijper, A. (2018) · 2018 IEEE 9th International Conference on Biometrics Theory, Applications and Systems (BTAS) · doi:10.1109/btas.2018.8698563 · Cited by 133
Automatic Generation and Detection of Visually Faultless Facial Morphs
Makrushin, A., Neubert, T., & Dittmann, J. (2017) · Proceedings of the 12th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications · doi:10.5220/0006131100390050 · Cited by 131
On the Effects of Image Alterations on Face Recognition Accuracy
Ferrara, M., Franco, A., & Maltoni, D. (2016) · Springer eBooks · doi:10.1007/978-3-319-28501-6_9 · Cited by 129
Fraudulent ID using face morphs: Experiments on human and automatic recognition
Robertson, D. J., Kramer, R. S. S., & Burton, A. M. (2017) · PLOS ONE · doi:10.1371/journal.pone.0173319 · Cited by 119
Psychophysics: Introduction to Its Perceptual, Neural, and Social Prospects
Stevens, S. S., & Marks, L. E. (2017) · Routledge eBooks · doi:10.4324/9781315127675 · Cited by 118
Investigating Predictors of Superior Face Recognition Ability in Police Super‐recognisers
Davis, J. P., Lander, K., Evans, R., & Jansari, A. (2016) · Applied Cognitive Psychology · doi:10.1002/acp.3260 · Cited by 111
Individual differences in face identity processing
McCaffery, J. M., Robertson, D. J., Young, A. W., & Burton, A. M. (2018) · Cognitive Research: Principles and Implications · doi:10.1186/s41235-018-0112-9 · Cited by 97
The origin of Vierordt's law: The experimental protocol matters
Glasauer, S., & Shi, Z. (2021) · PsyCh Journal · doi:10.1002/pchj.464 · Cited by 80
Unfamiliar face matching: Pairs out-perform individuals and provide a route to training
Dowsett, A. J., & Burton, A. M. (2015) · British Journal of Psychology · doi:10.1111/bjop.12103 · Cited by 77
Face Recognition Vendor Test (FRVT) part 4: MORPH - performance of automated face morph detection
Ngan, M. L., Grother, P. J., Hanaoka, K., & Kuo, J. (2020) · doi:10.6028/nist.ir.8292 · Cited by 76
Handbook of Digital Face Manipulation and Detection: From DeepFakes to Morphing Attacks
(2022) · Advances in Computer Vision and Pattern Recognition · doi:10.1007/978-3-030-87664-7 · Cited by 72
Self-citationFace Recognition Across the Imaging Spectrum
Bourlai, T. (2016) · Cambridge International Law Journal · doi:10.1007/978-3-319-28501-6 · Cited by 63
The Relationship between Task Performance and Functional Magnetic Resonance Imaging Response
Buračas, G. T., Fine, I., & Boynton, G. M. (2005) · Journal of Neuroscience · doi:10.1523/jneurosci.4476-04.2005 · Cited by 55
Elze, T. (2010) · PLoS ONE · doi:10.1371/journal.pone.0012792 · Cited by 52
Detecting morphed passport photos: a training and individual differences approach
Robertson, D. J., Mungall, A., Watson, D. G., Wade, K. A., Nightingale, S. J., & Butler, S. (2018) · Cognitive Research: Principles and Implications · doi:10.1186/s41235-018-0113-8 · Cited by 49 · cited 6×
“Furthermore, this design was applied to the morph detection training task in [6] and subsequently in [7].”
BackgroundMethodResultInfluentialFace morphing attacks: Investigating detection with humans and computers
Kramer, R. S. S., Mireku, M. O., Flack, T. R., & Ritchie, K. L. (2019) · Cognitive Research: Principles and Implications · doi:10.1186/s41235-019-0181-4 · Cited by 48
Subjective and Objective Evaluation of Deepfake Videos
Korshunov, P., & Marcel, S. (2021) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp39728.2021.9414258 · Cited by 44
von Békésy, G. (1947) · Acta Oto-Laryngologica · doi:10.3109/00016484709124225 · Cited by 34
Face Authenticity: An Overview of Face Manipulation Generation, Detection and RecognitionPreprint
Akhtar, Z., Dasgupta, D., & Banerjee, B. (2019) · SSRN Electronic Journal · doi:10.2139/ssrn.3419272 · Cited by 33
Decision strategies in the ABX (matching-to-sample) psychophysical task
Hautus, M. J., & Meng, X. (2002) · Perception & Psychophysics · doi:10.3758/bf03194559 · Cited by 30
Statistical Analysis of ABX Results Using Signal Detection Theory
Boley, J., & Lester, M. (2009) · Journal of the Audio Engineering Society · Cited by 30
Wirth, B. E., & Carbon, C.-C. (2017) · Journal of Experimental Psychology: Applied · doi:10.1037/xap0000114 · Cited by 28
An algorithm for automatic skin smoothing in digital portraits
Lee, C., Schramm, M. T., Boutin, M., & Allebach, J. P. (2009) · Proceedings - International Conference on Image Processing · doi:10.1109/icip.2009.5414430 · Cited by 27
Lawless, H. T., & Heymann, H. (2010) · Food science text series · doi:10.1007/978-1-4419-6488-5_4 · Cited by 27
Dhall, A., Sharma, G., Bhatt, R. B., & Khan, G. M. (2009) · Lecture notes in computer science · doi:10.1007/978-3-642-10520-3_69 · Cited by 25
The First 100 Milliseconds of a Face: On the Microgenesis of Early Face Processing
Carbon, C. (2011) · Perceptual and Motor Skills · doi:10.2466/07.17.22.pms.113.6.859-874 · Cited by 25
Analysis of Images, Social Networks and Texts
van Der Aalst, W., Ignatov, D. I., Khachay, M., et al. (2017) · Lecture notes in computer science · doi:10.1007/978-3-319-73013-4 · Cited by 25
Yang, Q., & Ng, M. L. (2017) · Elsevier eBooks · doi:10.1016/b978-0-08-101009-9.00005-8 · Cited by 22
Perceptual and computational detection of face morphing
Nightingale, S. J., Agarwal, S., & Farid, H. (2021) · Journal of Vision · doi:10.1167/jov.21.3.4 · Cited by 22
Face Morphing, a Modern Threat to Border Security: Recent Advances and Open Challenges
Pikoulis, E., Ioannou, Z.-M., Paschou, M., & Sakkopoulos, E. (2021) · Applied Sciences · doi:10.3390/app11073207 · Cited by 22
(2016) · doi:10.1016/c2012-0-01278-1 · Cited by 21
Weber, J. (1993) · Springer eBooks · doi:10.1007/978-3-642-50177-7 · Cited by 20
Weatherford, D. R., Roberson, D., & Erickson, W. B. (2021) · Cognitive Research Principles and Implications · doi:10.1186/s41235-021-00288-z · Cited by 20
Makrushin, A., Neubert, T., & Dittmann, J. (2019) · International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP) · doi:10.5220/0007378900002108 · Cited by 18
Bebis, G. N., Wang, J., Lindström, P., et al. (2009) · Lecture notes in computer science · doi:10.1007/978-3-642-10520-3 · Cited by 17
Individual differences in face perception and person recognition
Bruce, V., Bindemann, M., & Lander, K. (2018) · Cognitive Research: Principles and Implications · doi:10.1186/s41235-018-0109-4 · Cited by 16
Makrushin, A., Siegel, D., & Dittmann, J. (2020) · ACM Workshop on Information Hiding and Multimedia Security (IH&MMSec) · doi:10.1145/3369412.3395073 · Cited by 13
Greenaway, R. E. (2017) · Elsevier eBooks · doi:10.1016/b978-0-08-101009-9.00013-7 · Cited by 11
Billson, S. (2017) · Elsevier eBooks · doi:10.1016/b978-0-08-101009-9.00008-3 · Cited by 4
Research on Face Image Morphing Based on Automatic Feature Detection
Guanglai, T., Chen, C.-C., Sun, G., & Bin, S. (2020) · IEEE Eurasia Conference on Biomedical Engineering, Healthcare and Sustainability (ECBIOS) · doi:10.1109/ecbios50299.2020.9203749 · Cited by 3
Cimbal, W. (1913) · Springer eBooks · doi:10.1007/978-3-662-43030-9_7 · Cited by 2
Floor-Ladder Framework for Human Face Beautification
Novskaya, Y., Sun, R., Zhu, H., & Ma, L. (2017) · Lecture notes in computer science · doi:10.1007/978-3-319-73013-4_24 · Cited by 1
Kemp, R. (1997) · doi:10.1002/(sici)1099-0720(199706)11:3<211::aid-acp430>3.0.co;2-o
Research works printed but not linked: 18
- Bledsoe, 1966, The Model Method in Facial Recognition, 15, 2
- Green, 1966, Signal Detection Theory Psychophysics
- Richter, 2019, Faceapp Craze Reaches New Heights
- 2021, Leading Entertainment APP Titles in the Apple App Store in Spain
- 2020, Germany Bans Digital Doppelganger Passport Photos
- Wolberg, 1990, Digital Image Warping, 10662
- Kravets, 2017, Driver’s License Facial Recognition Tech Leads to 4,000 New York Arrests
- Boyd, 2021, CBP’s Facial Biometrics Program has Caught 26 Alleged Imposters
- Costello, 2018, New Facial Recognition Tech Catches First Impostor at DC Airport
- 2016, Information Technology—Biometric Presentation Attack Detection—Part 1: Framework
- 2020, Facial Recognition Statistics in Airports
- Xie, 2018, Council Post: The Future of Biometric Facial Recognition
- 2021, A Harmonized, Modular Reference System for all European Automated Border Crossing Points
- Ennis, 2012, Proper task instructions and the two-out-of-five-test, IFPress, 15, 3
- 2016, (EU) General Data Protection Regulation
- 2017, Information Technology—Biometric Presentation Attack Detection—Part 3: Testing and Reporting
- 2021, Fotorus—Camera & Photo Editor
- 2021, Instabeauty—Makeup Camera
Cited by
Deepfakes as a threat to a speaker and facial recognition: An overview of tools and attack vectors
Firc, A., Malinka, K., & Hanáček, P. (2023) · Heliyon · doi:10.1016/j.heliyon.2023.e15090 · Cited by 80
Human Performance in Deepfake Detection: A Systematic Review
Somoray, K., Miller, D. J., & Holmes, M. (2025) · Human Behavior and Emerging Technologies · doi:10.1155/hbe2/1833228 · Cited by 19
Challenges for automated face recognition systems
Busch, C. (2024) · Nature Reviews Electrical Engineering · doi:10.1038/s44287-024-00094-x · Cited by 12
Self-citationOverview of Research Progress of Digital Image Processing Technology
Huang, Y. (2022) · Journal of Physics Conference Series · doi:10.1088/1742-6596/2386/1/012034 · Cited by 11
Crowd-Powered Face Manipulation Detection: Fusing Human Examiner Decisions
Rathgeb, C., Nichols, R., Ibsen, M., Drozdowski, P., & Busch, C. (2022) · Proceedings - International Conference on Image Processing · doi:10.1109/icip46576.2022.9897841 · Cited by 10
Self-citation
Metadata
- Authors
- Nichols, R., Rathgeb, C., Drozdowski, P., & Busch, C.
- Affiliations
- Darmstadt University of Applied Sciences (DE)
- Venue
- IEEE Access
- Published
- 2022
- Volume
- 10
- Pages
- 31359-31376
- ISSN
- 2169-3536 (ISSN-L)
- OpenAlex
- W4226373926
- Cited by
- 16
- References
- 65
- Citations by year
- 2026: 3 · 2025: 5 · 2024: 2 · 2023: 4 · 2022: 2
- FWCI
- 1.06
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- IEEE Access (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- arXiv (Cornell University)
- Funders
- German Federal Ministry of Education and Research
- Hessian State Ministry for Higher Education, Research and the Arts within their joint support of the National Research Center for Applied Cybersecurity ATHENE
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Digital Media Forensic Detection
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Face recognition and analysis
- Life Sciences › Neuroscience › Cognitive Neuroscience › Face Recognition and Perception
- Keywords
- face swapping, face morphing, retouching, identity verification, biometric attacks, face recognition security, psychophysics
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.