CoChat

It's Trying Too Hard To Look Real: Deepfake Moderation Mistakes and Identity-Based Bias

Mink, Jaron, Wei, Miranda, Munyendo, Collins W., Hugenberg, Kurt, Kohno, Tadayoshi, Redmiles, Elissa M., Wang, Gang · 2024 · Proceedings of the CHI Conference on Human Factors in Computing Systems
Cited by 11

The DOI matches its Crossref record; 19 of the 19 references it cites were found in a registry or an index.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1145/3613904.3641999

Cites

19 of 19 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 19 references

Research works printed but not linked: 76

  1. Gordon Willard Allport, Kenneth Clark, and Thomas Pettigrew. 1954. The nature of prejudice. Addison-wesley Reading, MA.
  2. Carolina Are and Pam Briggs. 2023. The emotional and financial impact of de-platforming on creators at the margins. Social Media+ Society 9, 1 (2023), 20563051231155103.
  3. Emily Bazelon. 2013. How to Stop the Bullies. The Atlantic. https://www.theatlantic.com/magazine/archive/2013/03/how-to-stop-bullies/309217/. · doi:10.12968/sece.2013.7.1819
  4. 10.1007/978-3-319-67256-4_32 · DOI
  5. 10.1080/23268743.2021.1947883 · DOI · Paper page
  6. Danielle Blunt and Ariel Wolf. 2020. Erased: The impact of FOSTA-SESTA and the removal of Backpage on sex workers. Anti-trafficking review14 (2020), 117–121.
  7. 10.1002/mar.21480 · DOI · Paper page
  8. Virginia Braun and Victoria Clarke. 2012. Thematic analysis. Vol. 2. American Psychological Association, Washington D.C. 51–71 pages.
  9. Joy Buolamwini and Timnit Gebru. 2018. Gender shades: Intersectional accuracy disparities in commercial gender classification. In Proc. of FAccT.
  10. Pavlo Burda, Luca Allodi, and Nicola Zannone. 2020. Don’t Forget the Human: a Crowdsourced Approach to Automate Response and Containment Against Spear Phishing Attacks. In Proc. of EuroS&P Workshops.
  11. 10.1007/b97636 · DOI · Paper page
  12. Deanna D Caputo, Shari Lawrence Pfleeger, Jesse D Freeman, and M Eric Johnson. 2013. Going spear phishing: Exploring embedded training and awareness. In Proc. of IEEE Symposium on Security and Privacy.
  13. Rune Haubo B Christensen. 2018. Cumulative link models for ordinal regression with the R package ordinal. Submitted in J. Stat. Software 35 (2018).
  14. Elizabeth Clark, Tal August, Sofia Serrano, Nikita Haduong, Suchin Gururangan, and Noah A Smith. 2021. All Ghat’s ‘Human’ Is Not gGld: Evaluating Human Evaluation of Generated Text. In Proc. of ACL-IJCNLP.
  15. Jacob Cohen. 1960. A coefficient of agreement for nominal scales. Educational and psychological measurement 20, 1 (1960), 37–46.
  16. Juliet Corbin and Anselm Strauss. 2008. Basics of qualitative research: Techniques and procedures for developing grounded theory. Sage Publications, Inc, Thousand Oaks, CA, US.
  17. O. Corneille K. Hugenberg and J. Potter. 2007. Applying the attractor field model to social cognition: Perceptual discrimination is facilitated but memory is impaired for faces displaying evaluatively-congruent expressions.Journal of Personality and Social Psychology 93 (2007) 335–352. · DOI
  18. National Research Council 2004. Measuring racial discrimination. National Academies Press.
  19. 10.1080/10584609.2016.1227000 · DOI · Paper page
  20. 10.1145/3170427.3186320 · DOI
  21. Stacy Jo Dixon. 2023. Distribution of LinkedIn users worldwide as of January 2023, by age group. https://www.statista.com/statistics/273505/global-linkedin-age-group/.
  22. Stacy Jo Dixon. 2023. Gender distribution of social media audiences worldwide as of January 2023, by platform. https://www.statista.com/statistics/274828/gender-distribution-of-active-social-media-users-worldwide-by-platform/.
  23. Samuel Dooley, Ryan Downing, George Wei, Nathan Shankar, Bradon Thymes, Gudrun Thorkelsdottir, Tiye Kurtz-Miott, Rachel Mattson, Olufemi Obiwumi, Valeriia Cherepanova, 2021. Comparing human and machine bias in face recognition. arXiv preprint arXiv:2110.08396 (2021). · arXiv
  24. 10.1007/s11199-019-01052-w · DOI · Paper page
  25. Benjamin Edelman, Michael Luca, and Dan Svirsky. 2017. Racial discrimination in the sharing economy: Evidence from a field experiment. American economic journal: applied economics 9, 2 (2017), 1–22.
  26. Ziv Epstein, Sydney Levine, David G Rand, and Iyad Rahwan. 2020. Who gets credit for AI-generated art?Iscience 23, 9 (2020).
  27. Julian J Faraway. 2016. Extending the linear model with R: generalized linear, mixed effects and nonparametric regression models. CRC press.
  28. 10.1145/3392845 · DOI
  29. 10.1177/0894439320914853 · DOI · Paper page
  30. Brian Friedberg and Joan Donovan. 2019. On the Internet, Nobody Knows You’re a Bot: Pseudoanonymous Influence Operations and Networked Social Movements. Journal of Design and Science (2019).
  31. 10.1109/ICPR.2014.211 · DOI · Paper page
  32. Tarleton Gillespie. 2018. Custodians of the Internet: Platforms, content moderation, and the hidden decisions that shape social media. Yale University Press.
  33. 10.1109/MSP.2013.69 · DOI
  34. David Marvin Green, John A Swets, 1966. Signal detection theory and psychophysics. Vol. 1. Wiley New York.
  35. Peter Green Catriona MacLeod and Phillip Alday. 2015. simr: Power Analysis for Generalised Linear Mixed Models by Simulation. https://cran.r-project.org/package=simr. · DOI
  36. 10.1145/3479610 · DOI · Paper page
  37. Kurtis Haut, Caleb Wohn, Victor Antony, Aidan Goldfarb, Melissa Welsh, Dillanie Sumanthiran, Ji-ze Jang, Md Rafayet Ali, and Ehsan Hoque. 2021. Could you become more credible by being White? Assessing impact of race on credibility with deepfakes. arXiv preprint arXiv:2102.08054 (2021). · arXiv
  38. Madeline E Heilman. 2012. Gender stereotypes and workplace bias. Research in organizational Behavior 32 (2012), 113–135.
  39. 10.1109/CVPRW50498.2020.00329 · DOI
  40. 10.1007/s12671-019-01213-8 · DOI
  41. Joint Task Force on Cybersecurity Education. 2017. Cybersecurity Curricula 2017: Curriculum Guidelines for Post-Secondary Degree Programs in Cybersecurity. https://cybered.hosting.acm.org/wp-content/uploads/2018/02/newcover_csec2017.pdf. ACM/IEEE Joint Task Force Report (2017).
  42. 10.1145/1953163.1953173 · DOI
  43. 10.1109/ICASSP39728.2021.9414258 · DOI
  44. 10.1109/ICASSP39728.2021.9414258 · DOI
  45. 10.1109/MSP.2021.3120982 · DOI
  46. 10.1177/0956797617705399 · DOI
  47. Cameron Martel, Jennifer Allen, Gordon Pennycook, and David G Rand. 2022. Crowds can effectively identify misinformation at scale. Perspectives on Psychological Science (2022).
  48. 10.1016/j.jesp.2021.104199 · DOI · Paper page
  49. Ninareh Mehrabi, Fred Morstatter, Nripsuta Saxena, Kristina Lerman, and Aram Galstyan. 2021. A Survey on Bias and Fairness in Machine Learning. ACM Comput. Surv., Article 115 (July 2021), 35 pages.
  50. Meta. 2023. Community Standards: Manipulated Media. https://transparency.fb.com/policies/community-standards/manipulated-media/.
  51. Jaron Mink, Licheng Luo, Natã M. Barbosa, Olivia Figueira, Yang Wang, and Gang Wang. 2022. DeepPhish: Understanding User Trust Towards Artificially Generated Profiles in Online Social Networks. In Proc. of USENIX Security Symposium.
  52. 10.54501/jots.v1i4.62 · DOI
  53. Diana C Mutz. 2011. Population-based survey experiments. Princeton University Press.
  54. 10.1177/1461444818773059 · DOI · Paper page
  55. Bilal Naqvi, Kseniia Perova, Ali Farooq, Imran Makhdoom, Shola Oyedeji, and Jari Porras. 2023. Mitigation strategies against the phishing attacks: A systematic literature review. Computers & Security (2023), 103387.
  56. Martin T Orne. 2017. On the social psychology of the psychological experiment: With particular reference to demand characteristics and their implications. In Sociological methods. Routledge, 279–299.
  57. 10.1007/s13347-021-00485-0 · DOI
  58. 10.5244/C.29.41 · DOI · Paper page
  59. Thomas F Pettigrew and Linda R Tropp. 2006. A meta-analytic test of intergroup contact theory.Journal of personality and social psychology 90, 5 (2006), 751.
  60. 10.1017/CBO9780511844522.004 · doi:10.1017/cbo9780511844522.004
  61. Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. 2022. Hierarchical text-conditional image generation with clip latents. arXiv preprint arXiv:2204.06125 (2022). · arXiv
  62. Sarah T Roberts. 2014. Behind the screen: The hidden digital labor of commercial content moderation. University of Illinois at Urbana-Champaign.
  63. 10.1016/j.riob.2018.12.002 · DOI · Paper page
  64. 10.18653/v1/2022.naacl-main.431 · DOI
  65. 10.1109/MSP.2013.84 · DOI
  66. 10.1109/CVPR.2015.7298682 · DOI · Paper page
  67. Farhana Shahid, Srujana Kamath, Annie Sidotam, Vivian Jiang, Alexa Batino, and Aditya Vashistha. 2022. “It Matches My Worldview”: Examining Perceptions and Attitudes Around Fake Videos. In Proc. of CHI.
  68. Leron Solomon. 2015. Fair users or content abusers: The automatic flagging of non-infringing videos by content id on youtube. Hofstra L. Rev. 44 (2015), 237.
  69. 10.1007/s12369-018-0502-7 · DOI · Paper page
  70. Jack E Taylor, Guillaume A Rousselet, Christoph Scheepers, and Sara C Sereno. 2022. Rating norms should be calculated from cumulative link mixed effects models. Behavior Research Methods (2022), 1–22.
  71. 10.1145/3416061 · DOI
  72. 10.1145/3415231 · DOI · Paper page
  73. Rick Wash and Molly M Cooper. 2018. Who provides phishing training? facts, stories, and people like me. In Proc. of CHI.
  74. 10.1007/s11023-019-09506-6 · DOI · Paper page
  75. Leslie A. Zebrowitz. 2017. First Impressions From Faces. Current Directions in Psychological Science (2017).
  76. 10.1109/CVPR46437.2021.00222 · DOI · Paper page

Cited by

See all 11

Metadata

Authors
Mink, J., Wei, M., Munyendo, C. W., Hugenberg, K., Kohno, T., Redmiles, E. M., & Wang, G.
Affiliations
  • Computer Science, University of Illinois at Urbana-Champaign, United States
  • Paul G. Allen School of Computer Science & Engineering, University of Washington, United States
  • Computer Science, The George Washington University, United States
  • Psychological and Brain Sciences, Indiana University, United States
  • Computer Science, Georgetown University, United States
Venue
Proceedings of the CHI Conference on Human Factors in Computing Systems
Publisher
ACM
Published
2024-05-11
Pages
1-20
Cited by
11
References
19
Funders
National Science Foundation (1746047, 2030521, 2055233, 2206950, 2205171, 2207019)

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: It's Trying Too Hard To Look Real: Deepfake Moderation Mistakes and Identity-Based Bias. https://app.cochat.ai/papers/10.1145/3613904.3641999
The paper
Mink, J., Wei, M., Munyendo, C. W., Hugenberg, K., Kohno, T., Redmiles, E. M., & Wang, G. (2024). It's Trying Too Hard To Look Real: Deepfake Moderation Mistakes and Identity-Based Bias. Proceedings of the CHI Conference on Human Factors in Computing Systems. https://doi.org/10.1145/3613904.3641999

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.