It's Trying Too Hard To Look Real: Deepfake Moderation Mistakes and Identity-Based Bias
The DOI matches its Crossref record; 19 of the 19 references it cites were found in a registry or an index.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Since early-generation LLMs, humans have been consistently poor at distinguishing AI-generated text (Mink et al., 2024), audio (Warren et al., 2024;Barrington et al., 2025) images (Nightingale & Farid, 2022;Mink et al., 2024), videos (Groh et al., 2022), and social media profiles containing AI-generated media (Mink et al., 2024).”
From Open Challenges in Multi-Agent Security: Towards Secure Systems of Interacting AI Agents (2025) · cites this paper as Mink et al., 2024
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Social Capital: Prospects for a New Concept
Adler, P. S., & Kwon, S.-W. (2002) · The Academy of Management Review · doi:10.2307/4134367 · Cited by 9,665 · cited 1×
“Indirect economic harms may be incurred for general users from the loss of social capital (or the economic benefits that result from social relations [1]) achieved through social media [32, 79].”
BackgroundPower Analysis and Effect Size in Mixed Effects Models: A Tutorial
Brysbaert, M., & Stevens, M. (2018) · Journal of Cognition · doi:10.5334/joc.10 · Cited by 1,420 · cited 1×
“%) for the estimated effect size to be found [18].”
BackgroundExposing Deep Fakes Using Inconsistent Head Poses
Yang, X., Li, Y., & Lyu, S. (2019) · ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2019.8683164 · Cited by 1,196
An overview of online fake news: Characterization, detection, and discussion
Zhang, X., & Ghorbani, A. A. (2020) · Information Processing & Management · doi:10.1016/j.ipm.2019.03.004 · Cited by 997
The Creation and Detection of Deepfakes: A Survey
Mirsky, Y., & Lee, W. (2020) · arXiv · doi:10.48550/arxiv.2004.11138 · Cited by 863
The Creation and Detection of Deepfakes: A Survey
Mirsky, Y., & Lee, W. (2022) · ACM Computing Surveys · doi:10.1145/3425780 · Cited by 863
Deep learning for deepfakes creation and detection: A survey
Nguyen, T. T., Nguyen, Q. V. H., Nguyen, D. T., Nguyen, D. T., Huynh-The, T., Nahavandi, S., Nguyen, T. T., Pham, Q.-V., & Nguyen, C. M. (2022) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2022.103525 · Cited by 464
Deep Learning for Deepfakes Creation and Detection: A Survey
Nguyen, T. T., Nguyen, Q. V. H., Nguyen, D. T., Nguyen, D. T., Huynh-The, T., Nahavandi, S., Nguyen, T. T., Pham, Q.-V., & Nguyen, C. M. (2019) · arXiv · doi:10.48550/arxiv.1909.11573 · Cited by 464
AI-synthesized faces are indistinguishable from real faces and more trustworthy
Nightingale, S. J., & Farid, H. (2022) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2120481119 · Cited by 397
First I "like" it, then I hide it: Folk Theories of Social Feeds
Eslami, M., Karahalios, K., Sandvig, C., Vaccaro, K., Rickman, A., Hamilton, K., & Kirlik, A. (2016) · Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems · doi:10.1145/2858036.2858494 · Cited by 375
Research works printed but not linked: 76
- Gordon Willard Allport, Kenneth Clark, and Thomas Pettigrew. 1954. The nature of prejudice. Addison-wesley Reading, MA.
- Carolina Are and Pam Briggs. 2023. The emotional and financial impact of de-platforming on creators at the margins. Social Media+ Society 9, 1 (2023), 20563051231155103.
- Emily Bazelon. 2013. How to Stop the Bullies. The Atlantic. https://www.theatlantic.com/magazine/archive/2013/03/how-to-stop-bullies/309217/. · doi:10.12968/sece.2013.7.1819
- 10.1007/978-3-319-67256-4_32 · DOI
- 10.1080/23268743.2021.1947883 · DOI · Paper page
- Danielle Blunt and Ariel Wolf. 2020. Erased: The impact of FOSTA-SESTA and the removal of Backpage on sex workers. Anti-trafficking review14 (2020), 117–121.
- 10.1002/mar.21480 · DOI · Paper page
- Virginia Braun and Victoria Clarke. 2012. Thematic analysis. Vol. 2. American Psychological Association, Washington D.C. 51–71 pages.
- Joy Buolamwini and Timnit Gebru. 2018. Gender shades: Intersectional accuracy disparities in commercial gender classification. In Proc. of FAccT.
- Pavlo Burda, Luca Allodi, and Nicola Zannone. 2020. Don’t Forget the Human: a Crowdsourced Approach to Automate Response and Containment Against Spear Phishing Attacks. In Proc. of EuroS&P Workshops.
- 10.1007/b97636 · DOI · Paper page
- Deanna D Caputo, Shari Lawrence Pfleeger, Jesse D Freeman, and M Eric Johnson. 2013. Going spear phishing: Exploring embedded training and awareness. In Proc. of IEEE Symposium on Security and Privacy.
- Rune Haubo B Christensen. 2018. Cumulative link models for ordinal regression with the R package ordinal. Submitted in J. Stat. Software 35 (2018).
- Elizabeth Clark, Tal August, Sofia Serrano, Nikita Haduong, Suchin Gururangan, and Noah A Smith. 2021. All Ghat’s ‘Human’ Is Not gGld: Evaluating Human Evaluation of Generated Text. In Proc. of ACL-IJCNLP.
- Jacob Cohen. 1960. A coefficient of agreement for nominal scales. Educational and psychological measurement 20, 1 (1960), 37–46.
- Juliet Corbin and Anselm Strauss. 2008. Basics of qualitative research: Techniques and procedures for developing grounded theory. Sage Publications, Inc, Thousand Oaks, CA, US.
- O. Corneille K. Hugenberg and J. Potter. 2007. Applying the attractor field model to social cognition: Perceptual discrimination is facilitated but memory is impaired for faces displaying evaluatively-congruent expressions.Journal of Personality and Social Psychology 93 (2007) 335–352. · DOI
- National Research Council 2004. Measuring racial discrimination. National Academies Press.
- 10.1080/10584609.2016.1227000 · DOI · Paper page
- 10.1145/3170427.3186320 · DOI
- Stacy Jo Dixon. 2023. Distribution of LinkedIn users worldwide as of January 2023, by age group. https://www.statista.com/statistics/273505/global-linkedin-age-group/.
- Stacy Jo Dixon. 2023. Gender distribution of social media audiences worldwide as of January 2023, by platform. https://www.statista.com/statistics/274828/gender-distribution-of-active-social-media-users-worldwide-by-platform/.
- Samuel Dooley, Ryan Downing, George Wei, Nathan Shankar, Bradon Thymes, Gudrun Thorkelsdottir, Tiye Kurtz-Miott, Rachel Mattson, Olufemi Obiwumi, Valeriia Cherepanova, 2021. Comparing human and machine bias in face recognition. arXiv preprint arXiv:2110.08396 (2021). · arXiv
- 10.1007/s11199-019-01052-w · DOI · Paper page
- Benjamin Edelman, Michael Luca, and Dan Svirsky. 2017. Racial discrimination in the sharing economy: Evidence from a field experiment. American economic journal: applied economics 9, 2 (2017), 1–22.
- Ziv Epstein, Sydney Levine, David G Rand, and Iyad Rahwan. 2020. Who gets credit for AI-generated art?Iscience 23, 9 (2020).
- Julian J Faraway. 2016. Extending the linear model with R: generalized linear, mixed effects and nonparametric regression models. CRC press.
- 10.1145/3392845 · DOI
- 10.1177/0894439320914853 · DOI · Paper page
- Brian Friedberg and Joan Donovan. 2019. On the Internet, Nobody Knows You’re a Bot: Pseudoanonymous Influence Operations and Networked Social Movements. Journal of Design and Science (2019).
- 10.1109/ICPR.2014.211 · DOI · Paper page
- Tarleton Gillespie. 2018. Custodians of the Internet: Platforms, content moderation, and the hidden decisions that shape social media. Yale University Press.
- 10.1109/MSP.2013.69 · DOI
- David Marvin Green, John A Swets, 1966. Signal detection theory and psychophysics. Vol. 1. Wiley New York.
- Peter Green Catriona MacLeod and Phillip Alday. 2015. simr: Power Analysis for Generalised Linear Mixed Models by Simulation. https://cran.r-project.org/package=simr. · DOI
- 10.1145/3479610 · DOI · Paper page
- Kurtis Haut, Caleb Wohn, Victor Antony, Aidan Goldfarb, Melissa Welsh, Dillanie Sumanthiran, Ji-ze Jang, Md Rafayet Ali, and Ehsan Hoque. 2021. Could you become more credible by being White? Assessing impact of race on credibility with deepfakes. arXiv preprint arXiv:2102.08054 (2021). · arXiv
- Madeline E Heilman. 2012. Gender stereotypes and workplace bias. Research in organizational Behavior 32 (2012), 113–135.
- 10.1109/CVPRW50498.2020.00329 · DOI
- 10.1007/s12671-019-01213-8 · DOI
- Joint Task Force on Cybersecurity Education. 2017. Cybersecurity Curricula 2017: Curriculum Guidelines for Post-Secondary Degree Programs in Cybersecurity. https://cybered.hosting.acm.org/wp-content/uploads/2018/02/newcover_csec2017.pdf. ACM/IEEE Joint Task Force Report (2017).
- 10.1145/1953163.1953173 · DOI
- 10.1109/ICASSP39728.2021.9414258 · DOI
- 10.1109/ICASSP39728.2021.9414258 · DOI
- 10.1109/MSP.2021.3120982 · DOI
- 10.1177/0956797617705399 · DOI
- Cameron Martel, Jennifer Allen, Gordon Pennycook, and David G Rand. 2022. Crowds can effectively identify misinformation at scale. Perspectives on Psychological Science (2022).
- 10.1016/j.jesp.2021.104199 · DOI · Paper page
- Ninareh Mehrabi, Fred Morstatter, Nripsuta Saxena, Kristina Lerman, and Aram Galstyan. 2021. A Survey on Bias and Fairness in Machine Learning. ACM Comput. Surv., Article 115 (July 2021), 35 pages.
- Meta. 2023. Community Standards: Manipulated Media. https://transparency.fb.com/policies/community-standards/manipulated-media/.
- Jaron Mink, Licheng Luo, Natã M. Barbosa, Olivia Figueira, Yang Wang, and Gang Wang. 2022. DeepPhish: Understanding User Trust Towards Artificially Generated Profiles in Online Social Networks. In Proc. of USENIX Security Symposium.
- 10.54501/jots.v1i4.62 · DOI
- Diana C Mutz. 2011. Population-based survey experiments. Princeton University Press.
- 10.1177/1461444818773059 · DOI · Paper page
- Bilal Naqvi, Kseniia Perova, Ali Farooq, Imran Makhdoom, Shola Oyedeji, and Jari Porras. 2023. Mitigation strategies against the phishing attacks: A systematic literature review. Computers & Security (2023), 103387.
- Martin T Orne. 2017. On the social psychology of the psychological experiment: With particular reference to demand characteristics and their implications. In Sociological methods. Routledge, 279–299.
- 10.1007/s13347-021-00485-0 · DOI
- 10.5244/C.29.41 · DOI · Paper page
- Thomas F Pettigrew and Linda R Tropp. 2006. A meta-analytic test of intergroup contact theory.Journal of personality and social psychology 90, 5 (2006), 751.
- 10.1017/CBO9780511844522.004 · doi:10.1017/cbo9780511844522.004
- Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. 2022. Hierarchical text-conditional image generation with clip latents. arXiv preprint arXiv:2204.06125 (2022). · arXiv
- Sarah T Roberts. 2014. Behind the screen: The hidden digital labor of commercial content moderation. University of Illinois at Urbana-Champaign.
- 10.1016/j.riob.2018.12.002 · DOI · Paper page
- 10.18653/v1/2022.naacl-main.431 · DOI
- 10.1109/MSP.2013.84 · DOI
- 10.1109/CVPR.2015.7298682 · DOI · Paper page
- Farhana Shahid, Srujana Kamath, Annie Sidotam, Vivian Jiang, Alexa Batino, and Aditya Vashistha. 2022. “It Matches My Worldview”: Examining Perceptions and Attitudes Around Fake Videos. In Proc. of CHI.
- Leron Solomon. 2015. Fair users or content abusers: The automatic flagging of non-infringing videos by content id on youtube. Hofstra L. Rev. 44 (2015), 237.
- 10.1007/s12369-018-0502-7 · DOI · Paper page
- Jack E Taylor, Guillaume A Rousselet, Christoph Scheepers, and Sara C Sereno. 2022. Rating norms should be calculated from cumulative link mixed effects models. Behavior Research Methods (2022), 1–22.
- 10.1145/3416061 · DOI
- 10.1145/3415231 · DOI · Paper page
- Rick Wash and Molly M Cooper. 2018. Who provides phishing training? facts, stories, and people like me. In Proc. of CHI.
- 10.1007/s11023-019-09506-6 · DOI · Paper page
- Leslie A. Zebrowitz. 2017. First Impressions From Faces. Current Directions in Psychological Science (2017).
- 10.1109/CVPR46437.2021.00222 · DOI · Paper page
Cited by
Open Challenges in Multi-Agent Security: Towards Secure Systems of Interacting AI Agents
de Witt, C. S., Krawiecka, K., Krawczuk, I., Hagag, B., Anderson, W. L., Belcak, P., Bucknall, B., Cai, X., Chopra, A., Cohen, D., et al. (2025) · arXiv · doi:10.48550/arxiv.2505.02077 · Cited by 108
Self-citation
Metadata
- Authors
- Mink, J., Wei, M., Munyendo, C. W., Hugenberg, K., Kohno, T., Redmiles, E. M., & Wang, G.
- Affiliations
- Computer Science, University of Illinois at Urbana-Champaign, United States
- Paul G. Allen School of Computer Science & Engineering, University of Washington, United States
- Computer Science, The George Washington University, United States
- Psychological and Brain Sciences, Indiana University, United States
- Computer Science, Georgetown University, United States
- ORCID
- Venue
- Proceedings of the CHI Conference on Human Factors in Computing Systems
- Publisher
- ACM
- Published
- 2024-05-11
- Pages
- 1-20
- Cited by
- 11
- References
- 19
- Funders
- National Science Foundation (1746047, 2030521, 2055233, 2206950, 2205171, 2207019)
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: It's Trying Too Hard To Look Real: Deepfake Moderation Mistakes and Identity-Based Bias. https://app.cochat.ai/papers/10.1145/3613904.3641999
- The paper
- Mink, J., Wei, M., Munyendo, C. W., Hugenberg, K., Kohno, T., Redmiles, E. M., & Wang, G. (2024). It's Trying Too Hard To Look Real: Deepfake Moderation Mistakes and Identity-Based Bias. Proceedings of the CHI Conference on Human Factors in Computing Systems. https://doi.org/10.1145/3613904.3641999
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.