Generative AI Considered Harmful
Open accessThe DOI matches its Crossref record; 21 of the 21 references it cites were found in a registry or an index.
Abstract
The recent months have seen an explosion of interest, hype, and concern about generative AI, driven by the release of ChatGPT. In this article I seek to explicate some potential and actual harms of the engineering and use of generative AI such as ChatGPT. With this I also suggest a reframing for researchers with an interest in interaction. With this reframing I seek to provoke researchers to consider studying the settings of ChatGPT development and use as active sites of production. Research should focus on the organisational, technological and interactional practices and contexts in and through which generative AI and its outputs—harmful and otherwise—are produced, by whom, to what end, and with what consequences on societies.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜
Bender, E. M., Gebru, T., McMillan-Major, A., & Shmitchell, S. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445922 · Cited by 6,988
Chatting and cheating: Ensuring academic integrity in the era of ChatGPT
Cotton, D. R. E., Cotton, P. A., & Shipway, J. R. (2024) · Innovations in Education and Teaching International · doi:10.1080/14703297.2023.2190148 · Cited by 3,087 · cited 1×
“From concerns around false authorship becoming commonplace6, to researchers demonstrating getting a ChatGPT-written paper accepted for publication [7], I believe the discourse around ChatGPT has probably already done a good job “provoking” you—emphatically.”
BackgroundChatting and Cheating. Ensuring academic integrity in the era of ChatGPT
Cotton, D., Cotton, P., & Shipway, J. R. (2023) · doi:10.35542/osf.io/mrz8h · Cited by 3,079
Turkopticon: interrupting worker invisibility in amazon mechanical turk
Irani, L. C., & Silberman, M. S. (2013) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/2470654.2470742 · Cited by 798 · cited 2×
“There is a role to play for innovations around justice and wellbeing for AI workers; for instance tools and practices to provide counseling for AI workers, such as content moderators exposed to harmful content [1], and tools such as ‘Turkopticon’ that give a platform for AI workers’ voices where wasn’t one before [14].”
BackgroundVoice Interfaces in Everyday Life
Porcheron, M., Fischer, J. E., Reeves, S., & Sharples, S. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3174214 · Cited by 717 · cited 1×
“Often drawing on ethnographic and observational empirical approaches, such studies show just how technologies-in-use are interactionally embedded into everyday human conduct, including mobile phones [6], voice interfaces [19], and self-driving cars [5].”
BackgroundSelf-citationProceedings of the SIGCHI Conference on Human Factors in Computing Systems
(2009) · doi:10.1145/1518701 · Cited by 570
Human-centered design considered harmful
Norman, D. A. (2005) · Interactions · doi:10.1145/1070960.1070976 · Cited by 487 · cited 2×
“In the tradition of articles titled ‘[x] considered harmful’ [8, 17, 20], I want to refect on both the phenomenon and the concept used to describe it, namely that of ‘generative AI’, and propose a re-framing that hopefully makes it more productive for us researchers interested in conversation and…”
BackgroundThe Trouble with Autopilots: Assisted and Autonomous Driving on the Social Road
Brown, B., & Laurier, E. (2017) · Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3025453.3025462 · Cited by 186 · cited 1×
“technologies-in-use are interactionally embedded into everyday human conduct, including mobile phones [6], voice interfaces [19], and self-driving cars [5].”
BackgroundEthnography considered harmful
Crabtree, A., Rodden, T., Tolmie, P., & Button, G. (2009) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1518701.1518835 · Cited by 147
Research works printed but not linked: 2
- Andrew Arsht and Daniel Etcovitch. 2018. The human cost of online content moderation. Harvard Journal of Law and Technology (2018).
- Tolga Bolukbasi, Kai-Wei Chang, James Y Zou, Venkatesh Saligrama, and Adam T Kalai. 2016. Man is to computer programmer as woman is to homemaker? debiasing word embeddings. Advances in neural information processing systems 29 (2016).
Cited by
Han, A., Zhou, X., Cai, Z., Han, S., Ko, R., Corrigan, S., & Peppler, K. A. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642438 · Cited by 114
“These safeguards may include content fltering, moderation of user inputs, and continuous improvement of the chatbot’s response mechanisms to detect and handle inappropriate or harmful content [36, 81, 92].”
BackgroundCharting the Future of AI in Project-Based Learning: A Co-Design Exploration with Students
Zheng, C., Yuan, K., Guo, B., Hadi Mogavi, R., Peng, Z., Ma, S., & Ma, X. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642807 · Cited by 70
Charting the Future of AI in Project-Based Learning: A Co-Design Exploration with Students
Zheng, C., Yuan, K., Guo, B., Mogavi, R. H., Peng, Z., Ma, S., & Ma, X. (2024) · arXiv · doi:10.48550/arxiv.2401.14915 · Cited by 70
Generative AI in Marketing: Promises, Perils, and Public Policy Implications
Kumar, V., Kotler, P., Gupta, S., & Rajan, B. (2024) · Journal of Public Policy & Marketing · doi:10.1177/07439156241286499 · Cited by 96
“Research has identi fi ed various risks associated with GAI, including the displacement of jobs, violation of privacy, unauthorized utilization of content without giving credit to its creators, and even hallucinations (e.g., Fischer 2023), among other concerns.”
Promising the future, encoding the past: AI hype and public media imagery
Dežman, D. V. (2024) · AI and Ethics · doi:10.1007/s43681-024-00474-x · Cited by 56
“Recent months have seen “an explosion of interest, hype, and concern” [50] surrounding generative AI, epitomising the current AI hype wave.”
Towards a Diffractive Analysis of Prompt-Based Generative AI
Rajcic, N., Llano Rodriguez, M. T., & McCormack, J. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3641971 · Cited by 37
Can generative AI motivate management students? The role of perceived value and information literacy
Jose, E. M. K., Prasanna, A., Kushwaha, B. P., & Das, M. (2024) · The International Journal of Management Education · doi:10.1016/j.ijme.2024.101082 · Cited by 69
Towards interculturally adaptive conversational AI
Brandt, A., & Hazel, S. (2024) · Applied Linguistics Review · doi:10.1515/applirev-2024-0187 · Cited by 48
“…aspects of AI technologies replicate the prejudices which exist in society, for example racial and gender biases, whether this is in relation to machine learning (Mehrabi et al. 2021), large language models (LLMs, such as ChatGPT; Bender et al. 2021), or generative AI more broadly (Fischer 2023).”
BackgroundIs Resistance Futile?: Early Career Game Developers, Generative AI, and Ethical Skepticism
Boucher, J. D., Smith, G., & Telliel, Y. D. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3641889 · Cited by 30
Abumalloh, R. A., Nilashi, M., Ooi, K., Tan, G. W., & Chan, H. K. (2024) · Computers in Industry · doi:10.1016/j.compind.2024.104128 · Cited by 38
Hwang, A. H., Siy, J. O., Shelby, R. M., & Lentz, A. (2024) · Designing Interactive Systems Conference · doi:10.1145/3643834.3661555 · Cited by 42
“The general public has been increasingly involved in interrogating the potential of this emerging technology – including the potential risks it poses [20, 24, 26, 33, 43, 52, 95, 116].”
Background"Ronaldo's a poser!": How the Use of Generative AI Shapes Debates in Online Forums
Zeng, Y., Shi, Y., Huang, X., Nah, F. F., & Lc, R. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713829 · Cited by 34
“For example, an internally synthesized algorithm of Large Language Models (LLMs) could produce hallucinations [12, 62], which may act as a catalyst for the spread of misinformation in online forums [12].”
Lao, Y., Hirvonen, N., & Larsson, S. (2025) · Journal of Documentation · doi:10.1108/jd-01-2025-0007 · Cited by 38
Bridging Generations using AI-Supported Co-Creative Activities
Kim, C. Y., Sato, A. J., White, N. T., Ho, H.-R., Lee, C. P., Hwang, Y., & Mutlu, B. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713718 · Cited by 35
“Although users of these applications express concerns about their usability and potential to cause harm [25, 75], they are finding widespread societal adoption due to their potential benefits and ease of access.”
Wang, Z., Shen, L., Kuang, E., Zhang, S., & Fan, M. (2024) · Designing Interactive Systems Conference · doi:10.1145/3643834.3660703 · Cited by 27
“Another concern is the potential for bias or errors in AI-generated content, which not only poses risks in working scenarios but also increases the time for users to validate data [20, 29, 41, 76].”
BackgroundInterrogating AI: Characterizing Emergent Playful Interactions with ChatGPT
Nikghalb, M. R., & Cheng, J. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3711015 · Cited by 27
“Below, we discuss this taxonomy around two constructs that originally inspired our investigation: playful experiences and the concept of agency.”
Heaton, D., Clos, J., Nichele, E., & Fischer, J. E. (2024) · Personal and Ubiquitous Computing · doi:10.1007/s00779-024-01811-x · Cited by 21
“For example, Fischer examined the implications of generative AI systems, such as ChatGPT, and highlights associated risks including false authorship, unreliable advice, and job displacement in copywriting [65].”
BackgroundSelf-citationSun, X., Liu, Y., Bosch, J. A., & Li, Z. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3711014 · Cited by 18
“Additionally, the potential misuse of CUIs, such as spreading harmful or misleading information [52], raises critical ethical concerns.”
Oguine, O. C., Anuyah, O., Agha, Z., Melgarez, I., Garcia, A. A., & Badillo-Urquiola, K. A. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3757639 · Cited by 28
“…in HCI has evolved over time, with a focus on the myriad online risks faced by youth, including cyberbullying, sexting, and exposure to harmful content across digital platforms such as social media [21, 30, 104, 109], social virtual reality [74, 89], and more recently, generative AI [40, 77].”
Mastrianni, A., Twede, H., Sarcevic, A., Wander, J. D., Austin-Tse, C. A., Saponas, S., Rehm, H. L., Conard, A. M., & Hall, A. K. (2025) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3756326 · Cited by 14
Razi, A., Bouzoubaa, L. A., Pessianzadeh, A., Seberger, J. S., & Rezapour, R. (2025) · Research Square · doi:10.21203/rs.3.rs-5686084/v1 · Cited by 14
Mo, W., Singh, A., & Holloway, C. (2024) · International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS) · doi:10.1145/3663548.3675609 · Cited by 10
“Its tendency to hallucinate, producing non-factual information, might be tolerable in some domains [27] but could cause serious harm in critical healthcare and medicine contexts [90].”
Altamimi, A. (2024) · Discover Sustainability · doi:10.1007/s43621-024-00774-6 · Cited by 11
Engage and Mobilize! Understanding Evolving Patterns of Social Media Usage in Emergency Management
Purohit, H., Buntain, C., Hughes, A., Peterson, S., Lorini, V., & Castillo, C. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3710965 · Cited by 18
“…than replace it; in the context of life-threatening EM, technological solutions must support human-in-the-loop processes to mitigate associated techno-centric risks–e.g., AI hallucinations, biases, etc.–as these risks are increasingly studied in related safety-critical fields [19, 25, 31, 67].”
A Not So Chatty “Chatbot”: Co-designing to support First-Time Parents in South Africa and Portugal
Meoli, L., Nunes, F., Félix, B., da Silva, J. C., Ntinga, X., & Densmore, M. (2024) · ACM Conference on Conversational User Interfaces (CUI) · doi:10.1145/3640794.3665571 · Cited by 5
“Open domain nonspecific LLMs may present the risk of perpetuating inaccurate information from open internet sources [28] as well as hallucinating and providing false responses to users [9].”
BackgroundChatGPT as a Tool for Equitable Education in Engineering Classes
Ghosh, S. (2024) · ASEE Annual Conference & Exposition · doi:10.18260/1-2--48458 · Cited by 12
Okopnyi, P., Nordberg, O. E., & Guribye, F. (2024) · Halfway to the Future Symposium · doi:10.1145/3686169.3686184 · Cited by 8
“The GenUI vision suggests employing Large Language Models (LLMs), which have many inherent issues [3, 15].”
Generative AI, Ethics, and Language Learning
Matthews, B. (2025) · Springer eBooks · doi:10.1007/978-3-031-51447-0_94-1 · Cited by 4
Kanza, Y., Krishnamurthy, B., & Srivastava, D. (2024) · ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems (SIGSPATIAL GIS) · doi:10.1145/3678717.3691269 · Cited by 5
Generative AI, Ethics, and Language Learning
Matthews, B., & Mohamed, M. S. P. (2025) · Springer eBooks · doi:10.1007/978-3-031-51447-0_94-2 · Cited by 2
Using Generative Artificial Intelligence in University Teaching
Kaufmann, C., Schmiedel, T., & Christen, P. (2024) · Lecture notes in networks and systems · doi:10.1007/978-3-031-64836-6_35 · Cited by 2
Wang, L., Gao, Y., Anyi, C. L., & Goel, A. K. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3790279 · Cited by 7
“This also stimulates a fear of being replaced by a "better" AI clone in interpersonal relationships, echoing previous research in both work and intimate contexts [56, 102, 115, 174].”
Final Thoughts: Digital Humanities Looking at Generative AI
Aguiar, M., & Araújo, S. (2024) · Springer eBooks · doi:10.1007/978-3-031-48941-9_28 · Cited by 5
Label Engineering Methods for ML Systems
Mazumder, S., Banipal, I. S., Asthana, S., & Zhang, B. (2024) · Lecture notes in networks and systems · doi:10.1007/978-3-031-66336-9_33
Using Inquisitive AI Feedback to Enhance Formative Assessment in First-Year Engineering
de Freitas, C. C. S. (2025) · IEEE World Engineering Education Conference (EDUNINE) · doi:10.1109/edunine62377.2025.10981408
“Additionally, it is important to be mindful of AI limitations or even increase awareness regarding false authorship and plagiarism, hallucinations and misinformation, biases, and learning equity [39]–[41].”
How Do You Want to View This? Generative AI, Creative Ethos, and TTRPG Hobbyists
Boucher, J. D., Smith, G., & Telliel, Y. D. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3748630 · Cited by 2
A Case for the Dangers of Predatory Journals and IA Tools in Conjunction for Researchers
Rivera-Alvarado, E., & Guadamuz, S. (2025) · Smart innovation, systems and technologies · doi:10.1007/978-981-95-1365-9_17
Özyürek, H., & Türen, U. (2026) · SN Computer Science · doi:10.1007/s42979-026-04883-z
Ouaazki, A., Pignard-Cheynel, N., Bezençon, V., & Holzer, A. (2026) · Lecture notes in business information processing · doi:10.1007/978-3-032-18484-9_2
AI-Supported Collaborative Online Programs for Promoting Intercultural Communication
Mutsuddi, I. (2026) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-3373-6325-7.ch011
Automated Membership Inference via Prompt-Based Attacks in Generative Models
Gallo, D., Liguori, A., Ritacco, E., Caviglione, L., Durante, F., & Manco, G. (2026) · Machine Learning · doi:10.1007/s10994-026-07010-4
“Noisy or uncleaned data can degrade the performance and introduce unintended impacts for both software ecosystems and individuals (Fischer, 2023).”
Kim, J., Yu, M., Zhi, J., Milani, S., Cheng, J., Fan, X., Shen, H., & Forlizzi, J. (2026) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3788080 · Cited by 3
“However, the proliferation of GenAI adoption in content creation has introduced several critical societal challenges [27, 74, 75].”
Gengler, E. J. (2026) · Lecture notes in information systems and organisation · doi:10.1007/978-3-032-08486-6_9
Beyond Extraction? Generating Games with AI, Responsibly
Sharma, V., Mather, E., Mirza-Babaei, P., & Kumar, N. (2026) · Designing Interactive Systems Conference · doi:10.1145/3800645.3813025 · Cited by 2
“Some researchers challenge AI to be genuinely creative, contending that these systems largely recombine existing material in novel forms [8, 52, 77].”
Singh, S., & Martin-Hammond, A. M. (2026) · JMIR Aging · doi:10.2196/79740 · Cited by 2
“Examining harm is also a topic that has been explored among VAI technologies more generally [62,63].”
Future Imperfect? Evaluating Whether AI-Powered Web Accessibility Overlays are Worth the Hype
Hartman, P., & Gorichanaz, T. (2026) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3805689.3812259
Tyulina, N. (2026) · ACM Conference on Conversational User Interfaces (CUI) · doi:10.1145/3816046.3816313
“Previous work has documented epistemic harms [5], racial bias [22], systematic bias [14], bias toward disabled communities [15], risks of psychological manipulation [1], plagiarism and replication of stereotypes [13], and clear evidence of harm in the form of inappropriate sexual behavior [19].”
Gilbert, D. (2026) · Bulletin of the Council for Research in Music Education · doi:10.5406/21627223.250.1.03
Metadata
- Authors
- Fischer, J. E.
- Affiliations
- University of Nottingham (GB)
- Corresponding author
- Fischer, Joel E
- ORCID
- Fischer, Joel E
- Venue
- Proceedings of the 5th International Conference on Conversational User Interfaces
- Publisher
- ACM
- Published
- 2023-07-19
- Pages
- 1-5
- OpenAlex
- W4384520713
- Cited by
- 47
- References
- 18
- Citations by year
- 2026: 12 · 2025: 14 · 2024: 21
- FWCI
- 11.71
- Type
- conference paper
- Language
- en
- Open access
- Locations
- ACM Conference on Conversational User Interfaces (CUI) (publishedVersion)
- Repository@Nottingham (University of Nottingham) (submittedVersion, cc-by)
- Repository@Nottingham (University of Nottingham) (submittedVersion)
- Funders
- Engineering and Physical Sciences Research Council (EP/Y009800/1, EP/V00784X/1, V00784X/1)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Social Sciences › Arts and Humanities › History and Philosophy of Science › Cybernetics and Technology in Society
- Social Sciences › Business, Management and Accounting › Management Information Systems › Information Systems Theories and Implementation
- Keywords
- generative AI, ChatGPT, AI harms, interaction design, organisational practices, technological practice
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.