CoChat
Sign Up
Cited by 121Ethics and Social Impacts of AIGenerative Adversarial Networks and Image SynthesisAdversarial Robustness in Machine Learning

Typology of Risks of Generative Text-to-Image Models

Open access
Bird, Charlotte, Ungless, Eddie, Kasirzadeh, Atoosa·2023·Proceedings of the 2023 AAAI/ACM Conference on AI, Ethics, and Society
DOI 10.1145/3600211.3604722
Open this paper in CoChat — free accountOpen PDF

Abstract

This paper investigates the direct risks and harms associated with modern text-to-image generative models, such as DALL-E and Midjourney, through a comprehensive literature review. While these models offer unprecedented capabilities for generating images, their development and use introduce new types of risk that require careful consideration. Our review reveals significant knowledge gaps concerning the understanding and treatment of these risks despite some already being addressed. We offer a taxonomy of risks across six key stakeholder groups, inclusive of unexplored issues, and suggest future research directions. We identify 22 distinct risk types, spanning issues from data bias to malicious use. The investigation presented here is intended to enhance the ongoing discourse on responsible model development and deployment. By highlighting previously overlooked risks and gaps, it aims to shape subsequent research and governance initiatives, guiding them toward the responsible, secure, and ethically conscious evolution of text-to-image models.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1145/3600211.3604722

Cites

26 of 26 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • The rapid advancement of generative AI has been accompanied by growing societal concerns and numerous instances of misuse.

    “The rapid advancement of generative AI, marked by accelerated software and hardware innovation and a proliferation of novel applications, has been accompanied by growing societal concerns and numerous instances of misuse (Bender et al. 2021;Weidinger et al. 2022;Bird, Ungless, and Kasirzadeh 2023).”

    From Epistemic Injustice in Generative AI (2024) · cites this paper as Bird, Ungless, and Kasirzadeh 2023

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 110

  1. J. Ackermann and Minjun Li. 2022. High-Resolution Image Editing via Multi-Stage Blended Diffusion. ArXiv (2022). https://doi.org/10.48550/arXiv.2210.12965 · DOI
  2. Evgenios Agathokleous, Matthias C. Rillig, Josep Peñuelas, and Zhen Yu. 2023. One hundred impor- tant questions facing plant science derived using a large language model. Trends in Plant Science (2023). https://doi.org/10.1016/j.tplants.2023.06.008 · DOI
  3. John Akers, Gagan Bansal, Gabriel Cadamuro, Christine Chen, Quanze Chen, Lucy Lin, Phoebe Mulcaire, Ra- jalakshmi Nandakumar, Matthew Rockett, Lucy Simko, John Toman, Tongshuang Wu, Eric Zeng, Bill Zorn, and Franziska Roesner. 2019. Technology-Enabled Disinformation: Summary, Lessons, and Recommendations. https://doi.org/10.48550/arXiv.1812.09383 arXiv:1812.09383 [cs]. · DOI
  4. Yuntao Bai, Saurav Kadavath, Sandipan Kundu, Amanda Askell, Jackson Kernion, Andy Jones, Anna Chen, Anna Goldie, Azalia Mirhoseini, Cameron McKinnon, et al. 2022. Constitutional ai: Harmlessness from ai feedback. arXiv preprint arXiv:2212.08073 (2022). · arXiv
  5. David Baidoo-Anu and Leticia Owusu Ansah. 2023. Education in the era of generative artificial intelligence (AI): Understanding the potential benefits of ChatGPT in promoting teaching and learning. Available at SSRN 4337484 (2023).
  6. Hritik Bansal, Da Yin, Masoud Monajatipoor, and Kai-Wei Chang. 2022. How well can Text-to-Image Generative Models understand Ethical Natural Language Interventions? (2022). https://doi.org/10.48550/ARXIV.2210.15230 · DOI
  7. Iz Beltagy, Kyle Lo, and Arman Cohan. 2019. SciBERT: A pretrained language model for scientific text. arXiv preprint arXiv:1903.10676 (2019). · arXiv
  8. Federico Bianchi, Pratyusha Kalluri, Esin Durmus, Faisal Ladhak, M. Cheng, Debora Nozza, Tatsunori Hashimoto, Dan Jurafsky, James Y. Zou, and Aylin Caliskan. 2022. Easily Accessible Text-to-Image Generation Amplifies Demo- graphic Stereotypes at Large Scale. ArXiv (2022). https://doi.org/10.48550/arXiv.2211.03759 · DOI
  9. Abeba Birhane, William Isaac, Vinodkumar Prabhakaran, Mark Diaz, Madeleine Clare Elish, Iason Gabriel, and Shakir Mohamed. 2022. Power to the People? Opportunities and Challenges for Participatory AI. In Equity and Access in Algorithms, Mechanisms, and Optimization (EAAMO '22). Association for Computing Machinery, New York, NY, USA, 1-8. https://doi.org/10.1145/3551624.3555290 · DOI · Paper page
  10. Abeba Birhane, William Samuel Isaac, Vinodkumar Prabhakaran, Mark Díaz, Madeleine Clare Elish, Iason Gabriel, and Shakir Mohamed. 2022. Frameworks and Challenges to Participatory AI. https://arxiv.org/pdf/2209.07572.pdf · arXiv
  11. Abeba Birhane, Vinay Uday Prabhu, and Emmanuel Kahembwe. 2021. Multimodal datasets: misogyny, pornography, and malignant stereotypes. http://arxiv.org/abs/2110.01963 arXiv:2110.01963 [cs]. · arXiv
  12. Su Lin Blodgett, Solon Barocas, Hal Daumé III, and Hanna Wallach. 2020. Language (Technology) is Power: A Critical Survey of "Bias" in NLP. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, Online, 5454-5476. https://doi.org/10.18653/v1/2020.acl-main.485 · DOI · Paper page
  13. Su Lin Blodgett and Brendan O'Connor. 2017. Racial Disparity in Natural Language Processing: A Case Study of Social Media African-American English. arXiv:1707.00061 [cs] (Jun 2017). http://arxiv.org/abs/1707.00061 arXiv: 1707.00061. · arXiv
  14. Rishi Bommasani, Kathleen Creel, Ananya Kumar, Dan Jurafsky, and Percy Liang. 2022. Picking on the Same Person: Does Algorithmic Monoculture lead to Outcome Homogenization? https://openreview.net/forum?id=-H6kKm4DVo
  15. Elizabeth Bondi, Lily Xu, Diana Acosta-Navas, and Jackson A. Killian. 2021. Envisioning Communities: A Participatory Approach Towards AI for Social Good. In Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society (AIES '21). Association for Computing Machinery, New York, NY, USA, 425-436. https://doi.org/10.1145/3461702.3462612 · DOI · Paper page
  16. Manuel Brack, Patrick Schramowski, Felix Friedrich, Dominik Hintersdorf, and Kristian Kersting. 2022. The Stable Artist: Steering Semantics in Diffusion Latent Space. (2022). https://doi.org/10.48550/ARXIV.2212.06013 · DOI
  17. Joy Buolamwini and Timnit Gebru. 2018. Gender Shades: Intersectional Accuracy Disparities in Commercial Gen- der Classification. In Proceedings of the 1st Conference on Fairness, Accountability and Transparency. PMLR, 77-91. https://proceedings.mlr.press/v81/buolamwini18a.html ISSN: 2640-3498.
  18. Tiago Sérgio Cabral. 2020. Liability and artificial intelligence in the EU: Assessing the adequacy of the current Product Liability Directive. Maastricht Journal of European and Comparative Law 27, 5 (2020), 615-635.
  19. Nicholas Carlini, Jamie Hayes, Milad Nasr, Matthew Jagielski, Vikash Sehwag, Florian Tramèr, Borja Balle, Daphne Ippolito, and Eric Wallace. 2023. Extracting Training Data from Diffusion Models. arXiv:2301.13188 (Jan 2023). http://arxiv.org/abs/2301.13188 arXiv:2301.13188 [cs]. · arXiv
  20. Eva Cetinic and James She. 2022. Understanding and Creating Art with AI: Review and Outlook. ACM Transactions on Multimedia Computing, Communications, and Applications 18, 2 (Feb. 2022), 66:1-66:22. https://doi.org/10.1145/3475799 · DOI · Paper page
  21. Jaemin Cho, Abhay Zala, and Mohit Bansal. 2022. DALL-Eval: Probing the Reasoning Skills and Social Biases of Text-to-Image Generative Models. arXiv:2202.04053 (Nov 2022). http://arxiv.org/abs/2202.04053 arXiv:2202.04053 [cs]. · arXiv
  22. Mark Coeckelbergh. 2017. Can Machines Create Art? Philosophy & Technology 30, 3 (Sept. 2017), 285-303. https://doi.org/10.1007/s13347-016-0231-5 · DOI
  23. Mark Coeckelbergh. 2020. AI Ethics. MIT Press. Google-Books-ID: Gs_XDwAAQBAJ.
  24. Hao Dang, Feng Liu, Joel Stehouwer, Xiaoming Liu, and Anil Jain. 2020. On the Detection of Digital Face Manipu- lation. https://doi.org/10.48550/arXiv.1910.01717 arXiv:1910.01717 [cs]. · DOI
  25. Sunipa Dev, Masoud Monajatipoor, Anaelia Ovalle, Arjun Subramonian, Jeff Phillips, and Kai-Wei Chang. 2021. Harms of Gender Exclusivity and Challenges in Non-Binary Representation in Language Technologies. In Proceed- ings of the 2021 Conference on Empirical Methods in Natural Language Processing. Association for Computational Lin- guistics, Online and Punta Cana, Dominican Republic, 1968-1994. https://doi.org/10.18653/v1/2021.emnlp-main.150 · DOI
  26. Sunipa Dev, Emily Sheng, Jieyu Zhao, Aubrie Amstutz, Jiao Sun, Yu Hou, Mattie Sanseverino, Jiin Kim, Akihiro Nishi, Nanyun Peng, and Kai-Wei Chang. 2022. On Measures of Biases and Harms in NLP. In Findings of the Association for Computational Linguistics: AACL-IJCNLP 2022. Association for Computational Linguistics, Online only, 246-267. https://aclanthology.org/2022.findings-aacl.24
  27. Prafulla Dhariwal and Alexander Quinn Nichol. 2022. Diffusion Models Beat GANs on Image Synthesis. https://openreview.net/forum?id=AAWuCvzaVt
  28. Laurent Dinh, Jascha Sohl-Dickstein, and Samy Bengio. 2017. Density estimation using Real NVP. https://doi.org/10.48550/arXiv.1605.08803 arXiv:1605.08803 [cs, stat]. · DOI · Paper page
  29. E. Durmus, K. Nyugen, T. I. Liao, N. Schiefer, A. Askell, A. Bakhtin, C. Chen, Z. Hatfield-Dodds, D. Hernandez, N. Joseph, and L. Lovitt. 2023. Towards Measuring the Representation of Subjective Global Opinions in Language Models. arXiv preprint arXiv:2306.16388 (2023). · arXiv
  30. Tyna Eloundou, Sam Manning, Pamela Mishkin, and Daniel Rock. 2023. Gpts are gpts: An early look at the labor market impact potential of large language models. arXiv preprint arXiv:2303.10130 (2023). · arXiv
  31. Melanie Freeze, Mary Baumgartner, Peter Bruno, Jacob R. Gunderson, Joshua Olin, Morgan Quinn Ross, and Justine Szafran. 2021. Fake Claims of Fake News: Political Misinformation, Warnings, and the Tainted Truth Effect. Political Behavior 43, 4 (Dec. 2021), 1433-1465. https://doi.org/10.1007/s11109-020-09597-3 · DOI
  32. Stanislav Frolov, Tobias Hinz, Federico Raue, Jörn Hees, and Andreas Dengel. 2021. Adversarial Text-to-Image Synthesis: A Review. Neural Networks 144 (Dec. 2021), 187-209. https://doi.org/10.1016/j.neunet.2021.07.019 arXiv:2101.09983 [cs]. · DOI
  33. José Gamir-Ríos, Raquel Tarullo, and Miguel Ibáñez-Cuquerella. 2021. Multimodal disinformation about otherness on the internet . The spread of racist, xenophobic and Islamophobic fake news in 2020. Anàlisi (July 2021), 49-64. https://doi.org/10.5565/rev/analisi.3398> · DOI
  34. Samuel Gehman, Suchin Gururangan, Maarten Sap, Yejin Choi, and Noah A. Smith. 2020. RealTox- icityPrompts: Evaluating Neural Toxic Degeneration in Language Models. In Findings of the Association for Computational Linguistics: EMNLP 2020. Association for Computational Linguistics, Online, 3356-3369. https://doi.org/10.18653/v1/2020.findings-emnlp.301 · DOI · Paper page
  35. Avijit Ghosh and Genoveva Fossas. 2022. Can There be Art Without an Artist? (2022). https://doi.org/10.48550/ARXIV.2209.07667 · DOI
  36. Dimitrios Giomelakis, Olga Papadopoulou, Symeon Papadopoulos, and Andreas Veglis. 2021. Verification of News Video Content: Findings from a Study of Journalism Students. Journalism Practice (Aug. 2021), 1-30. https://doi.org/10.1080/17512786.2021.1965905 · DOI
  37. Seraphina Goldfarb-Tarrant, Eddie Ungless, Esma Balkir, and Su Lin Blodgett. 2023. This Prompt is Measuring <MASK>: Evaluating Bias Evaluation in Language Models. arXiv:2305.12757 [cs.CL] · arXiv
  38. Katelyn Golladay and Kristy Holtfreter. 2017. The Consequences of Identity Theft Victimiza- tion: An Examination of Emotional and Physical Health Outcomes. Victims & Offenders 12, 5 (Sept. 2017), 741-760. https://doi.org/10.1080/15564886.2016.1177766 Publisher: Routledge _eprint: https://doi.org/10.1080/15564886.2016.1177766. · DOI
  39. Philipp Hacker. 2022. The European AI Liability Directives-Critique of a Half-Hearted Approach and Lessons for the Future. arXiv preprint arXiv:2211.13960 (2022). · arXiv
  40. Ryuichiro Hataya, Han Bao, and Hiromi Arai. 2022. Will Large-scale Generative Models Corrupt Future Datasets? (2022). https://doi.org/10.48550/ARXIV.2211.08095 · DOI
  41. Natali Helberger and Nicholas Diakopoulos. 2023. ChatGPT and the AI Act. Internet Policy Review 12, 1 (2023).
  42. Dirk Helbing, Bruno S. Frey, Gerd Gigerenzer, Ernst Hafen, Michael Hagner, Yvonne Hofstetter, Jeroen van den Hoven, Roberto V. Zicari, and Andrej Zwitter. 2019. Will Democracy Survive Big Data and Artificial Intelligence? In Towards Digital Enlightenment: Essays on the Dark and Light Sides of the Digital Revolution, Dirk Helbing (Ed.). Springer International Publishing, Cham, 73-98. https://doi.org/10.1007/978-3-319-90869-4_7 · DOI · Paper page
  43. Jonathan Ho, Ajay Jain, and Pieter Abbeel. 2020. Denoising Diffusion Probabilistic Models. (June 2020). https://doi.org/10.48550/arXiv.2006.11239 · DOI · Paper page
  44. Stephanie Houde, Vera Liao, Jacquelyn Martino, Michael Muller, David Piorkowski, John Richards, Justin Weisz, and Yunfeng Zhang. 2020. Business (mis)Use Cases of Generative AI. http://arxiv.org/abs/2003.07679 arXiv:2003.07679 [cs]. · arXiv
  45. Ben Hutchinson, Jason Baldridge, and Vinodkumar Prabhakaran. 2022. Underspecification in Scene Description-to- Depiction Tasks. arXiv. https://doi.org/10.48550/ARXIV.2210.05815 · DOI
  46. Ivan Kapelyukh, Vitalis Vosylius, and Edward Johns. 2022. DALL-E-Bot: Introducing Web-Scale Diffusion Models to Robotics. arXiv:2210.02438 (Nov 2022). http://arxiv.org/abs/2210.02438 arXiv:2210.02438 [cs]. · arXiv
  47. Atoosa Kasirzadeh. 2022. Algorithmic Fairness and Structural Injustice: Insights from Feminist Political Philosophy. In AIES '22: Proceedings of the 2022 AAAI/ACM Conference on AI, Ethics, and Society. 349-356. https://doi.org/10.1145/3514094.3534188 · DOI
  48. Atoosa Kasirzadeh and Iason Gabriel. 2023. In conversation with Artificial Intelligence: aligning language models with human values. Philosophy & Technology 36, 2 (2023), 1-24. https://doi.org/10.1007/s13347-023-00606-x · DOI · Paper page
  49. Atoosa Kasirzadeh and Colin Klein. 2021. The ethical gravity thesis: Marrian levels and the persistence of bias in automated decision-making systems. In Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society. 618-626.
  50. Dilara Keküllüoğlu, Nadin Kökciyan, and Pınar Yolum. 2016. Strategies for Privacy Negotiation in Online Social Net- works. In Proceedings of the 1st International Workshop on AI for Privacy and Security. ACM, The Hague Netherlands, 1-8. https://doi.org/10.1145/2970030.2970035 · DOI
  51. Brandon Khoo, Raphaël C.-W. Phan, and Chern-Hong Lim. 2022. Deepfake attribution: On the source iden- tification of artificially generated images. WIREs Data Mining and Knowledge Discovery 12, 3 (2022), e1438. https://doi.org/10.1002/widm.1438 _eprint: https://onlinelibrary.wiley.com/doi/pdf/10.1002/widm.1438. · DOI
  52. Diederik P. Kingma and Max Welling. 2022. Auto-Encoding Variational Bayes. https://doi.org/10.48550/arXiv.1312.6114 arXiv:1312.6114 [cs, stat]. · DOI · Paper page
  53. Timur Kuran and Edward J. McCaffery. 2004. Expanding Discrimination Research: Beyond Ethnicity and to the Web*. Social Science Quarterly 85, 3 (2004), 713-730. https://doi.org/10.1111/j.0038-4941.2004.00241.x · DOI
  54. Nadin Kökciyan, Nefise Yaglikci, and Pinar Yolum. 2017. An Argumentation Approach for Resolving Pri- vacy Disputes in Online Social Networks. ACM Transactions on Internet Technology 17, 3 (Aug 2017), 1-22. https://doi.org/10.1145/3003434 · DOI
  55. Markus Langer, Daniel Oster, Timo Speith, Holger Hermanns, Lena Kästner, Eva Schmidt, Andreas Sesing, and Kevin Baum. 2021. What do we want from Explainable Artificial Intelligence (XAI)? -A stakeholder perspective on XAI and a conceptual model guiding interdisciplinary XAI research. Artificial Intelligence 296 (Jul 2021), 103473. https://doi.org/10.1016/j.artint.2021.103473 · DOI · Paper page
  56. M. Asher Lawson, Shikhar Anand, and Hemant Kakkar. 2023. Tribalism and tribulations: The social costs of not sharing fake news. Journal of Experimental Psychology: General 152, 3 (2023), 611-631. https://doi.org/10.1037/xge0001374 Place: US Publisher: American Psychological Association. · DOI
  57. David M. J. Lazer, Matthew A. Baum, Yochai Benkler, Adam J. Berinsky, Kelly M. Greenhill, Filippo Menczer, Miriam J. Metzger, Brendan Nyhan, Gordon Pennycook, David Rothschild, Michael Schudson, Steven A. Sloman, Cass R. Sun- stein, Emily A. Thorson, Duncan J. Watts, and Jonathan L. Zittrain. 2018. The science of fake news. Science 359, 6380 (March 2018), 1094-1096. https://doi.org/10.1126/science.aao2998 Publisher: American Association for the Advancement of Science. · DOI · Paper page
  58. Percy Liang, Rishi Bommasani, Tony Lee, Dimitris Tsipras, Dilara Soylu, Michihiro Yasunaga, Yian Zhang, Deepak Narayanan, Yuhuai Wu, Ananya Kumar, et al. 2022. Holistic evaluation of language models. arXiv preprint arXiv:2211.09110 (2022). · arXiv
  59. Juniper Lovato, Laurent Hébert-Dufresne, Jonathan St-Onge, Randall Harp, Gabriela Salazar Lopez, Sean P. Rogers, Ijaz Ul Haq, and Jeremiah Onaolapo. 2022. Diverse Misinformation: Impacts of Human Biases on Detection of Deepfakes on Networks. (2022). https://doi.org/10.48550/ARXIV.2210.10026 Publisher: arXiv Version Number: 2. · DOI
  60. Nina Markl. 2022. Mind the data gap(s): Investigating power in speech and language datasets. In Proceedings of the Second Workshop on Language Technology for Equality, Diversity and Inclusion. Association for Computational Linguistics, Dublin, Ireland, 1-12. https://doi.org/10.18653/v1/2022.ltedi-1.1 · DOI
  61. Sherin Mathews, Shivangee Trivedi, Amanda House, Steve Povolny, and Celeste Fralick. 2023. An explainable deepfake detection framework on a novel unconstrained dataset. Complex & Intelligent Systems (Jan. 2023). https://doi.org/10.1007/s40747-022-00956-7 · DOI
  62. Raphaël Millière. 2022. Adversarial Attacks on Image Generation With Made-Up Words. ArXiv (2022). https://doi.org/10.48550/arXiv.2208.04135 · DOI
  63. Raphaël Millière. 2022. Deep learning and synthetic media. Synthese 200, 3 (May 2022), 231. https://doi.org/10.1007/s11229-022-03739-2 · DOI
  64. Jakob Mökander, Prathm Juneja, David S Watson, and Luciano Floridi. 2022. The US Algorithmic Accountability Act of 2022 vs. The EU Artificial Intelligence Act: what can they learn from each other? Minds and Machines 32, 4 (2022), 751-758.
  65. Paul Nemitz. 2018. Constitutional democracy and technology in the age of artificial intelligence. Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences 376, 2133 (Oct. 2018), 20180089. https://doi.org/10.1098/rsta.2018.0089 Publisher: Royal Society. · DOI · Paper page
  66. Alexis Newton and Kaustubh Dhole. 2023. Is AI Art Another Industrial Revolution in the Making? (2023). https://doi.org/10.48550/ARXIV.2301.05133 · DOI
  67. Alex Nichol, Prafulla Dhariwal, Aditya Ramesh, Pranav Shyam, Pamela Mishkin, Bob McGrew, Ilya Sutskever, and Mark Chen. 2022. GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models. https://doi.org/10.48550/arXiv.2112.10741 arXiv:2112.10741 [cs]. · DOI · Paper page
  68. Aaron van den Oord, Nal Kalchbrenner, Oriol Vinyals, Lasse Espeholt, Alex Graves, and Koray Kavukcuoglu. 2016. Conditional Image Generation with PixelCNN Decoders. https://doi.org/10.48550/arXiv.1606.05328 arXiv:1606.05328 [cs]. · DOI
  69. Jonas Oppenlaender. 2022. The Creativity of Text-to-Image Generation. In Proceedings of the 25th International Academic Mindtrek Conference (Academic Mindtrek '22). Association for Computing Machinery, New York, NY, USA, 192-202. https://doi.org/10.1145/3569219.3569352 · DOI
  70. Eli Pariser. 2011. The Filter Bubble: How the New Personalized Web Is Changing What We Read and How We Think. Penguin. Google-Books-ID: wcalrOI1YbQC.
  71. Seongbeom Park, Suhong Moon, and Jinkyu Kim. 2022. Judge, Localize, and Edit: Ensuring Visual Commonsense Morality for Text-to-Image Generation. https://doi.org/10.48550/arXiv.2212.03507 arXiv:2212.03507 [cs]. · DOI
  72. Muxin Pu, Meng Yi Kuan, Nyee Thoang Lim, Chun Yong Chong, and Mei Kuan Lim. 2022. Fairness Evaluation in Deepfake Detection Models using Metamorphic Testing. In 2022 IEEE/ACM 7th International Workshop on Metamor- phic Testing (MET). 7-14. https://doi.org/10.1145/3524846.3527337 · DOI
  73. Jielin Qiu, Yi Zhu, Xingjian Shi, Florian Wenzel, Zhiqiang Tang, Ding Zhao, Bo Li, and Mu Li. 2022. Are Multimodal Models Robust to Image and Text Perturbations? (2022). https://doi.org/10.48550/ARXIV.2212.08044 · DOI · Paper page
  74. Organizers Of Queerinai, Anaelia Ovalle, Arjun Subramonian, Ashwin Singh, Claas Voelcker, Danica J. Sutherland, Davide Locatelli, Eva Breznik, Filip Klubicka, Hang Yuan, Hetvi J, Huan Zhang, Jaidev Shriram, Kruno Lehman, Luca Soldaini, Maarten Sap, Marc Peter Deisenroth, Maria Leonor Pacheco, Maria Ryskina, Martin Mundt, Milind Agarwal, Nyx Mclean, Pan Xu, A Pranav, Raj Korpan, Ruchira Ray, Sarah Mathew, Sarthak Arora, St John, Tanvi Anand, Vishakha Agrawal, William Agnew, Yanan Long, Zijie J. Wang, Zeerak Talat, Avijit Ghosh, Nathaniel Dennler, Michael Noseworthy, Sharvani Jha, Emi Baylor, Aditya Joshi, Natalia Y. Bilenko, Andrew Mcnamara, Raphael Gontijo-Lopes, Alex Markham, Evyn Dong, Jackie Kay, Manu Saraswat, Nikhil Vytla, and Luke Stark. 2023. Queer In AI: A Case Study in Community-Led Participatory AI. In Proceedings of the 2023 ACM Conference on Fairness, Ac- countability, and Transparency (FAccT '23). Association for Computing Machinery, New York, NY, USA, 1882-1895. https://doi.org/10.1145/3593013.3594134 · DOI
  75. Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. 2021. Learning Transferable Visual Models From Natural Language Supervision. https://doi.org/10.48550/arXiv.2103.00020 arXiv:2103.00020 [cs]. · DOI · Paper page
  76. Mehul S Raval, Mohendra Roy, and Minoru Kuribayashi. 2022. Survey on Vision based Fake News Detection and its Impact Analysis. 2022 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC) (Nov. 2022), 1837-1841. https://doi.org/10.23919/APSIPAASC55919.2022.9980089 Conference Name: 2022 Asia Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC) ISBN: 9786165904773 Place: Chiang Mai, Thailand Publisher: IEEE. · DOI
  77. Scott E. Reed, Zeynep Akata, Xinchen Yan, L. Logeswaran, B. Schiele, and Honglak Lee. 2016. Generative Adversarial Text to Image Synthesis. ArXiv (May 2016). https://www.semanticscholar.org/paper/6c7f040a150abf21dbcefe1f22e0f98fa184f41a
  78. Karen Renaud, Merrill Warkentin, and George Westerman. 2023. From ChatGPT to HackGPT: Meeting the Cyberse- curity Threat of Generative AI. MIT Sloan Management Review.
  79. Yasmim Mendes Rocha, Gabriel Acácio de Moura, Gabriel Alves Desidério, Carlos Henrique de Oliveira, Fran- cisco Dantas Lourenço, and Larissa Deadame de Figueiredo Nicolete. 2021. The impact of fake news on social media and its influence on health during the COVID-19 pandemic: a systematic review. Zeitschrift Fur Gesundheitswis- senschaften (Oct. 2021), 1-10. https://doi.org/10.1007/s10389-021-01658-z · DOI
  80. Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily Denton, Seyed Kamyar Seyed Ghasemipour, Burcu Karagol Ayan, S. Sara Mahdavi, Rapha Gontijo Lopes, Tim Salimans, Jonathan Ho, David J. Fleet, and Mohammad Norouzi. 2022. Photorealistic Text-to-Image Diffusion Models with Deep Language Under- standing. http://arxiv.org/abs/2205.11487 arXiv:2205.11487 [cs]. · arXiv
  81. Joni Salminen, Soon-gyo Jung, Shammur Chowdhury, and Bernard J. Jansen. 2020. Analyzing Demographic Bias in Artificially Generated Facial Pictures. In Extended Abstracts of the 2020 CHI Conference on Human Fac- tors in Computing Systems (CHI EA '20). Association for Computing Machinery, New York, NY, USA, 1-8. https://doi.org/10.1145/3334480.3382791 · DOI
  82. Emily Saltz, Claire R Leibowicz, and Claire Wardle. 2021. Encounters with Visual Misinformation and Labels Across Platforms: An Interview and Diary Study to Inform Ecosystem Approaches to Misinformation Interventions. In Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems (CHI EA '21). Association for Computing Machinery, New York, NY, USA, 1-6. https://doi.org/10.1145/3411763.3451807 · DOI
  83. Pamela Samuelson. 2023. Legal Challenges to Generative AI, Part I. Commun. ACM 66, 7 (2023), 20-23.
  84. Glorin Sebastian. 2023. Do ChatGPT and other AI chatbots pose a cybersecurity risk?: An exploratory study. Inter- national Journal of Security and Privacy in Pervasive Computing (IJSPPC) 15, 1 (2023), 1-11.
  85. Oshani Seneviratne. 2022. Blockchain for Social Good: Combating Misinformation on the Web with AI and Blockchain. 14th ACM Web Science Conference 2022 (June 2022), 435-442. https://doi.org/10.1145/3501247.3539016 Conference Name: WebSci '22: 14th ACM Web Science Conference 2022 ISBN: 9781450391917 Place: Barcelona Spain Publisher: ACM. · DOI
  86. Sachith Seneviratne, Damith Senanayake, Sanka Rasnayaka, Rajith Vidanaarachchi, and Jason Thompson. 2022. DALLE-URBAN: Capturing the urban design expertise of large text to image transformers. arXiv:2208.04139 (Oct 2022). http://arxiv.org/abs/2208.04139 arXiv:2208.04139 [cs]. · arXiv
  87. Jia Wen Seow, Mei Kuan Lim, Raphaël C.W. Phan, and Joseph K. Liu. 2022. A comprehensive overview of Deepfake: Generation, detection, datasets, and opportunities. Neurocomputing 513 (Nov. 2022), 351-371. https://doi.org/10.1016/j.neucom.2022.09.135 · DOI · Paper page
  88. Boaz Shmueli, Jan Fell, Soumya Ray, and Lun-Wei Ku. 2021. Beyond Fair Pay: Ethical Implications of NLP Crowd- sourcing. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computa- tional Linguistics: Human Language Technologies. Association for Computational Linguistics, Online, 3758-3769. https://doi.org/10.18653/v1/2021.naacl-main.295 · DOI
  89. Jascha Sohl-Dickstein, Eric A. Weiss, Niru Maheswaranathan, and Surya Ganguli. 2015. Deep Unsupervised Learning using Nonequilibrium Thermodynamics. https://doi.org/10.48550/arXiv.1503.03585 arXiv:1503.03585 [cond-mat, q-bio, stat]. · DOI · Paper page
  90. Gowthami Somepalli, Vasu Singla, Micah Goldblum, Jonas Geiping, and Tom Goldstein. 2022. Diffusion Art or Digital Forgery? Investigating Data Replication in Diffusion Models. https://doi.org/10.48550/arXiv.2212.03860 arXiv:2212.03860 [cs]. · DOI
  91. Lukas Struppek, Dominik Hintersdorf, and Kristian Kersting. 2022. Rickrolling the Artist: Injecting Invisible Back- doors into Text-Guided Image Generation Models. (2022). https://doi.org/10.48550/ARXIV.2211.02408 · DOI
  92. Naofse Mac Sweeney. 2009. Beyond Ethnicity: The Overlooked Diversity of Group Identities. Journal of Mediter- ranean Archaeology 22, 1 (Jun 2009), 101-126. https://doi.org/10.1558/jmea.v22i1.101 · DOI
  93. Nenad Tomasev, Jonathan Leader Maynard, and Iason Gabriel. 2022. Manifestations of Xenophobia in AI Systems. (2022). https://doi.org/10.48550/ARXIV.2212.07877 Publisher: arXiv Version Number: 1. · DOI
  94. Chad M. Topaz, Jude Higdon, Avriel Epps-Darling, Ethan Siau, Harper Kerkhoff, Shivani Mendiratta, and Eric Young. 2022. Race-and gender-based under-representation of creative contributors: art, fashion, film, and music. Humanities and Social Sciences Communications 9, 11 (Jun 2022), 1-11. https://doi.org/10.1057/s41599-022-01239-9 · DOI
  95. Inga Trauthig. 2022. WhatsApp, Misinformation, and Latino Political Discourse in the U.S. https://techpolicy.press/whatsapp-misinformation-and-latino-political-discourse-in-the-u-s/
  96. Eddie L. Ungless, Björn Ross, and Anne Lauscher. 2023. Stereotypes and Smut: The (Mis)representation of Non- cisgender Identities by Text-to-Image Models. arXiv:2305.17072 (May 2023). http://arxiv.org/abs/2305.17072 arXiv:2305.17072 [cs]. · arXiv
  97. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention Is All You Need. (June 2017). https://doi.org/10.48550/arXiv.1706.03762 · DOI · Paper page
  98. Michael Veale, Kira Matus, and Robert Gorwa. 2023. AI and Global Governance: Modalities, Rationales, Tensions. An- nual Review of Law and Social Science 19 (2023). https://doi.org/10.1146/annurev-lawsocsci-020223-040749 Review in Advance first posted online on June 28, 2023. (Changes may still occur before final publication.). · DOI
  99. Luisa Verdoliva. 2020. Media Forensics and DeepFakes: an overview. http://arxiv.org/abs/2001.06564 arXiv:2001.06564 [cs]. · arXiv
  100. Nikhil Vyas, Sham Kakade, and Boaz Barak. 2023. Provable Copyright Protection for Generative Models. arXiv:2302.10870 (Feb 2023). http://arxiv.org/abs/2302.10870 arXiv:2302.10870 [cs, stat]. · arXiv
  101. Angelina Wang, Alexander Liu, Ryan Zhang, Anat Kleiman, Leslie Kim, Dora Zhao, Iroha Shirai, Arvind Narayanan, and Olga Russakovsky. 2022. REVISE: A Tool for Measuring and Mitigating Bias in Visual Datasets. International Journal of Computer Vision 130, 7 (Jul 2022), 1790-1810. https://doi.org/10.1007/s11263-022-01625-5 · DOI
  102. Su Wang, Chitwan Saharia, Ceslee Montgomery, Jordi Pont-Tuset, Shai Noy, Stefano Pellegrini, Yasumasa Onoe, Sarah Laszlo, David J. Fleet, Radu Soricut, Jason Baldridge, Mohammad Norouzi, Peter Anderson, and William Chan. 2022. Imagen Editor and EditBench: Advancing and Evaluating Text-Guided Image Inpainting. https://doi.org/10.48550/arXiv.2212.06909 arXiv:2212.06909 [cs]. · DOI
  103. Laura Weidinger, John Mellor, Maribeth Rauh, Conor Griffin, Jonathan Uesato, Po-Sen Huang, Myra Cheng, Mia Glaese, Borja Balle, Atoosa Kasirzadeh, Zac Kenton, Sasha Brown, Will Hawkins, Tom Stepleton, Courtney Biles, Abeba Birhane, Julia Haas, Laura Rimell, Lisa Anne Hendricks, William Isaac, Sean Legassick, Geoffrey Irving, and Ia- son Gabriel. 2021. Ethical and social risks of harm from Language Models. https://doi.org/10.48550/arXiv.2112.04359 arXiv:2112.04359 [cs]. · DOI · Paper page
  104. Justin D. Weisz, Michael Muller, Jessica He, and Stephanie Houde. 2023. Toward General Design Principles for Generative AI Applications. (2023). https://doi.org/10.48550/ARXIV.2301.05578 Publisher: arXiv Version Number: 1. · DOI · Paper page
  105. Mika Westerlund. 2019. The Emergence of Deepfake Technology: A Review. Technology Innovation Management Review 9, 11 (2019), 40-53. https://doi.org/10.22215/timreview/1282 Place: Ottawa Publisher: Talent First Network. · DOI · Paper page
  106. David Gray Widder, Dawn Nafus, Laura Dabbish, and James Herbsleb. 2022. Limits and Possibilities for &#x201c;Ethical AI&#x201d; in Open Source: A Study of Deepfakes. In 2022 ACM Conference on Fairness, Ac- countability, and Transparency (FAccT '22). Association for Computing Machinery, New York, NY, USA, 2035-2046. https://doi.org/10.1145/3531146.3533779 · DOI
  107. Robert Wolfe, Yiwei Yang, Bill Howe, and Aylin Caliskan. 2022. Contrastive Language-Vision AI Models Pre- trained on Web-Scraped Multimodal Data Exhibit Sexual Objectification Bias. arXiv:2212.11261 (Dec 2022). http://arxiv.org/abs/2212.11261 arXiv:2212.11261 [cs]. · arXiv
  108. Ying Xu, Philipp Terhörst, Kiran Raja, and Marius Pedersen. 2022. A Comprehensive Analysis of AI Biases in DeepFake Detection With Massively Annotated Databases. (2022). https://doi.org/10.48550/ARXIV.2208.05845 Publisher: arXiv Version Number: 1. · DOI
  109. Jiahui Yu, Yuanzhong Xu, Jing Yu Koh, Thang Luong, Gunjan Baid, Zirui Wang, Vijay Vasudevan, Alexander Ku, Yinfei Yang, Burcu Karagol Ayan, Ben Hutchinson, Wei Han, Zarana Parekh, Xin Li, Han Zhang, Jason Baldridge, and Yonghui Wu. 2022. Scaling Autoregressive Models for Content-Rich Text-to-Image Generation. (2022). https://doi.org/10.48550/ARXIV.2206.10789 Publisher: arXiv Version Number: 1. · DOI · Paper page
  110. Ning Yu, Vladislav Skripniuk, Sahar Abdelnabi, and Mario Fritz. 2020. Artificial Fingerprinting for Generative Mod- els: Rooting Deepfake Attribution in Training Data. (July 2020). https://doi.org/10.48550/arXiv.2007.08457 · DOI

Cited by

Sorted by citations

Authors
Bird, C., Ungless, E., & Kasirzadeh, A.
Affiliations
  • University of Edinburgh (GB)
  • The Alan Turing Institute (GB)
ORCID
Venue
Proceedings of the 2023 AAAI/ACM Conference on AI, Ethics, and Society
Publisher
ACM
Published
2023-08-08
Pages
396-410
ISSN
3065-8365 (ISSN-L)
DOI
10.1145/3600211.3604722
OpenAlex
W4386246859
Cited by
121
References
26
Citations by year
2026: 24 · 2025: 58 · 2024: 34 · 2023: 4 · 2022: 1
FWCI
30.04
Type
conference paper
Language
en
Open access
Licence
Locations
  • Proceedings of the AAAI/ACM Conference on AI Ethics and Society (publishedVersion, cc-by)
  • arXiv (Cornell University) (submittedVersion, cc-by)
  • Edinburgh Research Explorer (University of Edinburgh) (submittedVersion, cc-by)
  • arXiv (Cornell University) (cc-by)
Funders
  • AH/X007146/1 (UKRI)
  • UKRI (EP/S022481/)
Topics
  • Social Sciences › Safety Research › Ethics and Social Impacts of AI
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Generative Adversarial Networks and Image Synthesis
  • Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
Keywords
text-to-image generative models, AI risks, risk taxonomy, DALL-E, Midjourney, data bias, malicious use of AI, responsible AI development, AI governance, stakeholder risk

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review