An Examination of Fairness of AI Models for Deepfake Detection
Open accessAbstract
Recent studies have demonstrated that deep learning models can discriminate based on protected classes like race and gender. In this work, we evaluate bias present in deepfake datasets and detection models across protected subgroups. Using facial datasets balanced by race and gender, we examine three popular deepfake detectors and find large disparities in predictive performances across races, with up to 10.7% difference in error rate between subgroups. A closer look reveals that the widely used FaceForensics++ dataset is overwhelmingly composed of Caucasian subjects, with the majority being female Caucasians. Our investigation of the racial distribution of deepfakes reveals that the methods used to create deepfakes as positive training signals tend to produce ``irregular" faces - when a person’s face is swapped onto another person of a different race or gender. This causes detectors to learn spurious correlations between the foreground faces and fakeness. Moreover, when detectors are trained with the Blended Image (BI) dataset from Face X-Rays, we find that those detectors develop systematic discrimination towards certain racial subgroups, primarily female Asians.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Trinh et al. is the first to evaluate biases in existing deepfake datasets and detection models across protected groups.
“The work in [14] is the first to evaluate biases in existing deepfake datasets and detection models across protected groups.”
From Improving Fairness in Deepfake Detection (2024) · cites this paper as [14]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
FaceForensics++: Learning to Detect Manipulated Facial Images
Rossler, A., Cozzolino, D., Verdoliva, L., Riess, C., Thies, J., & Niessner, M. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00009 · Cited by 3,314 · see citing works
Equality of Opportunity in Supervised Learning
Hardt, M., Price, E., & Srebro, N. (2016) · arXiv · doi:10.48550/arxiv.1610.02413 · Cited by 2,536 · see citing works
Gebru, T., Morgenstern, J., Vecchione, B., Vaughan, J. W., Wallach, H., Daumé, H., & Crawford, K. (2018) · arXiv · doi:10.48550/arxiv.1803.09010 · Cited by 2,474
Gebru, T., Morgenstern, J., Vecchione, B., Vaughan, J. W., Wallach, H., III, H. D., & Crawford, K. (2021) · Communications of the ACM · doi:10.1145/3458723 · Cited by 2,469 · see citing works
MesoNet: a Compact Facial Video Forgery Detection Network
Afchar, D., Nozick, V., Yamagishi, J., & Echizen, I. (2018) · 2018 IEEE International Workshop on Information Forensics and Security (WIFS) · doi:10.1109/wifs.2018.8630761 · Cited by 1,872 · see citing works
Age Progression/Regression by Conditional Adversarial Autoencoder
Zhang, Z., Song, Y., & Qi, H. (2017) · Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2017.463 · Cited by 1,480 · see citing works
Deepfakes and beyond: A Survey of face manipulation and fake detection
Tolosana, R., Vera-Rodriguez, R., Fierrez, J., Morales, A., & Ortega-Garcia, J. (2020) · Information Fusion · doi:10.1016/j.inffus.2020.06.014 · Cited by 1,237 · see citing works
Face X-Ray for More General Face Forgery Detection
Li, L., Bao, J., Zhang, T., Yang, H., Chen, D., Wen, F., & Guo, B. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00505 · Cited by 1,211 · see citing works
Exposing DeepFake Videos By Detecting Face Warping Artifacts
Li, Y., & Lyu, S. (2018) · arXiv · doi:10.48550/arxiv.1811.00656 · Cited by 785 · see citing works
DeeperForensics-1.0: A Large-Scale Dataset for Real-World Face Forgery Detection
Jiang, L., Li, R., Wu, W., Qian, C., & Loy, C. C. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00296 · Cited by 599 · see citing works
Global Texture Enhancement for Fake Face Detection in the Wild
Liu, Z., Qi, X., & Torr, P. H. S. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00808 · Cited by 401 · see citing works
What Makes Fake Images Detectable? Understanding Properties that Generalize
Chai, L., Bau, D., Lim, S.-N., & Isola, P. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58574-7_7 · Cited by 357 · see citing works
Racial faces in the wild: Reducing racial bias by information maximization adaptation network
Wang, M., Deng, W., Hu, J., Tao, X., & Huang, Y. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00078 · Cited by 353 · see citing works
FakeSpotter: A Simple yet Robust Baseline for Spotting AI-Synthesized Fake Faces
Wang, R., Juefei-Xu, F., Ma, L., Xie, X., Huang, Y., Wang, J., & Liu, Y. (2020) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2020/476 · Cited by 278 · see citing works
Self-citationDelayed Impact of Fair Machine Learning
Liu, L. T., Dean, S., Rolf, E., Simchowitz, M., & Hardt, M. (2019) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2019/862 · Cited by 236 · see citing works
ForensicTransfer: Weakly-supervised Domain Adaptation for Forgery Detection
Cozzolino, D., Thies, J., Rössler, A., Riess, C., Nießner, M., & Verdoliva, L. (2018) · arXiv · doi:10.48550/arxiv.1812.02510 · Cited by 221 · see citing works
Evading Deepfake-Image Detectors with White- and Black-Box Attacks
Carlini, N., & Farid, H. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw50498.2020.00337 · Cited by 170 · see citing works
Interpretable and Trustworthy Deepfake Detection via Dynamic Prototypes
Trinh, L., Tsang, M., Rambhatla, S., & Liu, Y. (2021) · 2021 IEEE Winter Conference on Applications of Computer Vision (WACV) · doi:10.1109/wacv48630.2021.00202 · Cited by 115 · see citing works
Self-citationFake Face Detection Methods: Can They Be Generalized?
Khodabakhsh, A., Ramachandra, R., Raja, K., Wasnik, P., & Busch, C. (2018) · 2018 International Conference of the Biometrics Special Interest Group (BIOSIG) · doi:10.23919/biosig.2018.8553251 · Cited by 108 · see citing works
Flagged first, then by citations
Research works printed but not linked: 6
- Tolga Bolukbasi, Kai-Wei Chang, James Y Zou, Venkatesh Saligrama, and Adam T Kalai. Man is to com- puter programmer as woman is to homemaker? debiasing word embeddings. In NIPS, 2016.
- Buolamwini and Gebru, 2018] Joy Buolamwini and Timnit Gebru. Gender shades: Intersectional accuracy disparities in commercial gender classification. In ACM FAccT, 2018.
- Celis et al., 2019] L Elisa Celis, Sayash Kapoor, Farnood Salehi, and Nisheeth Vishnoi. Controlling polarization in personaliza- tion: An algorithmic framework. In Proceedings of the confer- ence on fairness, accountability, and transparency, 2019.
- Mirsky and Lee, 2021] Yisroel Mirsky and Wenke Lee. The cre- ation and detection of deepfakes: A survey. ACM Computing Surveys (CSUR), 54(1):1-41, 2021.
- Margaret Mitchell, Simone Wu, Andrew Zaldivar, Parker Barnes, Lucy Vasserman, Ben Hutchinson, Elena Spitzer, Inioluwa Deborah Raji, and Timnit Gebru. Model cards for model reporting. In Proceedings of the conference on fairness, accountability, and transparency, pages 220-229, 2019.
- Cristian Vaccari and Andrew Chad- wick. Deepfakes and disinformation: Exploring the impact of synthetic political video on deception, uncertainty, and trust in news. Social Media + Society, 6(1):2056305120903408, 2020.
Cited by
Sorted by citations
Masood, M., Nawaz, M., Malik, K. M., Javed, A., Irtaza, A., & Malik, H. (2023) · Applied Intelligence · doi:10.1007/s10489-022-03766-z · Cited by 543 · see citing works
Detecting Deepfakes with Self-Blended Images
Shiohara, K., & Yamasaki, T. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01816 · Cited by 503 · see citing works
Non-Consensual Synthetic Intimate Imagery: Prevalence, Attitudes, and Knowledge in 10 Countries
Umbach, R., Henry, N., Beard, G. F., & Berryessa, C. M. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642382 · Cited by 73 · see citing works
Improving Fairness in Deepfake Detection
(2024) · doi:10.1109/wacv57701.2024.00459 · Cited by 64 · see citing works
GBDF: Gender Balanced DeepFake Dataset Towards Fair DeepFake Detection
Nadimpalli, A. V., & Rattani, A. (2023) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-37742-6_25 · Cited by 46 · see citing works
- Authors
- Trinh, L., & Liu, Y.
- Affiliations
- University of Southern California (US)
- Corresponding author
- Trinh, Loc
- ORCID
- Liu, Yan
- Venue
- Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence
- Published
- 2021-08
- Pages
- 567-574
- OpenAlex
- W3190501006
- Cited by
- 44
- References
- 19
- Citations by year
- 2026: 7 · 2025: 10 · 2024: 15 · 2023: 7 · 2022: 5
- FWCI
- 2.21
- Type
- conference paper
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- International Joint Conference on Artificial Intelligence (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- Defense Advanced Research Projects Agency
- Advanced Research Projects Agency
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Digital Media Forensic Detection
- Social Sciences › Safety Research › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Face recognition and analysis
- Keywords
- deepfake detection, algorithmic fairness, algorithmic bias, racial bias, gender bias, protected classes, FaceForensics++, spurious correlations, face swapping
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
