Generative AI Considered Harmful
Open accessThe DOI matches its Crossref record; 21 of the 21 references it cites were found in a registry or an index.
Abstract
The recent months have seen an explosion of interest, hype, and concern about generative AI, driven by the release of ChatGPT. In this article I seek to explicate some potential and actual harms of the engineering and use of generative AI such as ChatGPT. With this I also suggest a reframing for researchers with an interest in interaction. With this reframing I seek to provoke researchers to consider studying the settings of ChatGPT development and use as active sites of production. Research should focus on the organisational, technological and interactional practices and contexts in and through which generative AI and its outputs—harmful and otherwise—are produced, by whom, to what end, and with what consequences on societies.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜
Bender, E. M., Gebru, T., McMillan-Major, A., & Shmitchell, S. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445922 · Cited by 6,988
Chatting and cheating: Ensuring academic integrity in the era of ChatGPT
Cotton, D. R. E., Cotton, P. A., & Shipway, J. R. (2024) · Innovations in Education and Teaching International · doi:10.1080/14703297.2023.2190148 · Cited by 3,087 · cited 1×
“From concerns around false authorship becoming commonplace6, to researchers demonstrating getting a ChatGPT-written paper accepted for publication [7], I believe the discourse around ChatGPT has probably already done a good job “provoking” you—emphatically.”
BackgroundChatting and Cheating. Ensuring academic integrity in the era of ChatGPT
Cotton, D., Cotton, P., & Shipway, J. R. (2023) · doi:10.35542/osf.io/mrz8h · Cited by 3,079
Turkopticon: interrupting worker invisibility in amazon mechanical turk
Irani, L. C., & Silberman, M. S. (2013) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/2470654.2470742 · Cited by 798 · cited 2×
“There is a role to play for innovations around justice and wellbeing for AI workers; for instance tools and practices to provide counseling for AI workers, such as content moderators exposed to harmful content [1], and tools such as ‘Turkopticon’ that give a platform for AI workers’ voices where wasn’t one before [14].”
BackgroundVoice Interfaces in Everyday Life
Porcheron, M., Fischer, J. E., Reeves, S., & Sharples, S. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3174214 · Cited by 717 · cited 1×
“Often drawing on ethnographic and observational empirical approaches, such studies show just how technologies-in-use are interactionally embedded into everyday human conduct, including mobile phones [6], voice interfaces [19], and self-driving cars [5].”
BackgroundSelf-citationProceedings of the SIGCHI Conference on Human Factors in Computing Systems
(2009) · doi:10.1145/1518701 · Cited by 570
Human-centered design considered harmful
Norman, D. A. (2005) · Interactions · doi:10.1145/1070960.1070976 · Cited by 487 · cited 2×
“In the tradition of articles titled ‘[x] considered harmful’ [8, 17, 20], I want to refect on both the phenomenon and the concept used to describe it, namely that of ‘generative AI’, and propose a re-framing that hopefully makes it more productive for us researchers interested in conversation and…”
BackgroundThe Trouble with Autopilots: Assisted and Autonomous Driving on the Social Road
Brown, B., & Laurier, E. (2017) · Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3025453.3025462 · Cited by 186 · cited 1×
“technologies-in-use are interactionally embedded into everyday human conduct, including mobile phones [6], voice interfaces [19], and self-driving cars [5].”
BackgroundEthnography considered harmful
Crabtree, A., Rodden, T., Tolmie, P., & Button, G. (2009) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1518701.1518835 · Cited by 147
Research works printed but not linked: 2
- Andrew Arsht and Daniel Etcovitch. 2018. The human cost of online content moderation. Harvard Journal of Law and Technology (2018).
- Tolga Bolukbasi, Kai-Wei Chang, James Y Zou, Venkatesh Saligrama, and Adam T Kalai. 2016. Man is to computer programmer as woman is to homemaker? debiasing word embeddings. Advances in neural information processing systems 29 (2016).
Cited by
Han, A., Zhou, X., Cai, Z., Han, S., Ko, R., Corrigan, S., & Peppler, K. A. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642438 · Cited by 114
“These safeguards may include content fltering, moderation of user inputs, and continuous improvement of the chatbot’s response mechanisms to detect and handle inappropriate or harmful content [36, 81, 92].”
BackgroundCharting the Future of AI in Project-Based Learning: A Co-Design Exploration with Students
Zheng, C., Yuan, K., Guo, B., Hadi Mogavi, R., Peng, Z., Ma, S., & Ma, X. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642807 · Cited by 70
Charting the Future of AI in Project-Based Learning: A Co-Design Exploration with Students
Zheng, C., Yuan, K., Guo, B., Mogavi, R. H., Peng, Z., Ma, S., & Ma, X. (2024) · arXiv · doi:10.48550/arxiv.2401.14915 · Cited by 70
Generative AI in Marketing: Promises, Perils, and Public Policy Implications
Kumar, V., Kotler, P., Gupta, S., & Rajan, B. (2024) · Journal of Public Policy & Marketing · doi:10.1177/07439156241286499 · Cited by 96
“Research has identi fi ed various risks associated with GAI, including the displacement of jobs, violation of privacy, unauthorized utilization of content without giving credit to its creators, and even hallucinations (e.g., Fischer 2023), among other concerns.”
Promising the future, encoding the past: AI hype and public media imagery
Dežman, D. V. (2024) · AI and Ethics · doi:10.1007/s43681-024-00474-x · Cited by 56
“Recent months have seen “an explosion of interest, hype, and concern” [50] surrounding generative AI, epitomising the current AI hype wave.”
Metadata
- Authors
- Fischer, J. E.
- Affiliations
- University of Nottingham (GB)
- Corresponding author
- Fischer, Joel E
- ORCID
- Fischer, Joel E
- Venue
- Proceedings of the 5th International Conference on Conversational User Interfaces
- Publisher
- ACM
- Published
- 2023-07-19
- Pages
- 1-5
- OpenAlex
- W4384520713
- Cited by
- 47
- References
- 18
- Citations by year
- 2026: 12 · 2025: 14 · 2024: 21
- FWCI
- 11.71
- Type
- conference paper
- Language
- en
- Open access
- Locations
- ACM Conference on Conversational User Interfaces (CUI) (publishedVersion)
- Repository@Nottingham (University of Nottingham) (submittedVersion, cc-by)
- Repository@Nottingham (University of Nottingham) (submittedVersion)
- Funders
- Engineering and Physical Sciences Research Council (EP/Y009800/1, EP/V00784X/1, V00784X/1)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Social Sciences › Arts and Humanities › History and Philosophy of Science › Cybernetics and Technology in Society
- Social Sciences › Business, Management and Accounting › Management Information Systems › Information Systems Theories and Implementation
- Keywords
- generative AI, ChatGPT, AI harms, interaction design, organisational practices, technological practice
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Generative AI Considered Harmful. https://app.cochat.ai/papers/10.1145/3571884.3603756
- The paper
- Fischer, J. E. (2023). Generative AI Considered Harmful. Proceedings of the 5th International Conference on Conversational User Interfaces. https://doi.org/10.1145/3571884.3603756
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.