On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜
Open accessThe DOI matches its Crossref record; 156 of the 159 references it cites were found in a registry or an index, and 3 are not checked yet.
Abstract
The past 3 years of work in NLP have been characterized by the development and deployment of ever larger language models, especially for English. BERT, its variants, GPT-2/3, and others, most recently Switch-C, have pushed the boundaries of the possible both through architectural innovations and through sheer size. Using these pretrained models and the methodology of fine-tuning them for specific tasks, researchers have extended the state of the art on a wide array of tasks as measured by leaderboards on specific benchmarks for English. In this paper, we take a step back and ask: How big is too big? What are the possible risks associated with this technology and what paths are available for mitigating those risks? We provide recommendations including weighing the environmental and financial costs first, investing resources into curating and carefully documenting datasets rather than ingesting everything on the web, carrying out pre-development exercises evaluating how the planned approach fits into research and development goals and supports stakeholder values, and encouraging research directions beyond ever larger language models.
Checked 2026-11-22 · 3 references not checked yet
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The text we found for this paper looks like another document, so the self-check wasn’t run.
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325 · cited 5×
“[39] in particular noted that training on a large dataset and fine-tuning for specific tasks leads to strictly increasing results on the GLUE tasks [138] for English as the hyperparameters of the model were increased.”
BackgroundMethodResultInfluentialGlove: Global Vectors for Word Representation
Pennington, J., Socher, R., & Manning, C. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1162 · Cited by 34,071
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372 · cited 4×
“documents linked to from Reddit [25], plus Wikipedia and a collection”
BackgroundMethodInfluentialRoBERTa: A Robustly Optimized BERT Pretraining Approach
Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., & Stoyanov, V. (2019) · arXiv · doi:10.48550/arxiv.1907.11692 · Cited by 16,515
Distilling the Knowledge in a Neural Network
Hinton, G., Vinyals, O., & Dean, J. (2015) · arXiv · doi:10.48550/arxiv.1503.02531 · Cited by 14,204 · cited 1×
“[26, 58], quantization [118, 153], factorized embedding parameterization and cross-layer parameter sharing [70], and progressive module replacing [146].”
Deep Contextualized Word Representations
Peters, M., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., & Zettlemoyer, L. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1202 · Cited by 11,983
Proceedings of the 24th international conference on Machine learning
(2007) · doi:10.1145/1273496 · Cited by 11,630
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., & Liu, P. J. (2019) · arXiv · doi:10.48550/arxiv.1910.10683 · Cited by 10,929
The Mathematical Theory of Communication
Shannon, C. E., Weaver, W., & Wiener, N. (1950) · Physics Today · doi:10.1063/1.3067010 · Cited by 9,801
Research works printed but not linked: 48
- [4] American Psychological Association. 2019. Discrimination: What it is, and how to cope. https://www.apa.org/topics/discrimination (2019). Google Scholar
- 10.18653/v1 · DOI
- 10.18653/v1 · DOI
- [15] Ruha Benjamin. 2019. Race After Technology: Abolitionist Tools for the New Jim Code. Polity Press, Cambridge, UK. Crossref Google Scholar
- [16] Elettra Bietti and Roxana Vatanparast. 2020. Data Waste. Harvard International Law Journal 61 (2020). Google Scholar
- 10.18653/v1 · DOI
- 10.18653/v1 · DOI
- 10.1016/0010-0277(86) · doi:10.1016/0010-0277(86
- [38] Terrance de Vries, Ishan Misra, Changhan Wang, and Laurens van der Maaten. 2019. Does object recognition work for everyone?. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. 52--59. Google Scholar
- 10.18653/v1 · DOI
- [42] Ethan Fast, Tina Vachovsky, and Michael Bernstein. 2016. Shirtless and Dangerous: Quantifying Linguistic Signals of Gender Bias in an Online Fiction Writing Community. In Proceedings of the International AAAI Conference on Web and Social Media, Vol. 10. Google Scholar
- 10.18653/v1 · DOI
- [45] Darja Fišer, Ruihong Huang, Vinodkumar Prabhakaran, Rob Voigt, Zeerak Waseem, and Jacqueline Wernimont (Eds.). 2018. Proceedings of the 2nd Workshop on Abusive Language Online (ALW2). Association for Computational Linguistics, Brussels, Belgium. https://www.aclweb.org/anthology/W18-5100 Google Scholar
- [50] Batya Friedman, Peter H. Kahn, Jr., and Alan Borning. 2006. Value sensitive design and information systems. In Human-Computer Interaction in Management Information Systems: Foundations, P Zhang and D Galletta (Eds.). M. E. Sharpe, Armonk NY, 348--372. Google Scholar
- 10.18653/v1 · DOI
- 10.18653/v1 · DOI
- 10.18653/v1 · DOI
- [67] Brendan Kennedy, Drew Kogon, Kris Coombs, Joseph Hoover, Christina Park, Gwenyth Portillo-Wightman, Aida Mostafazadeh Davani, Mohammad Atari, and Morteza Dehghani. 2018. A typology and coding manual for the study of hate-based rhetoric. PsyArXiv. July 18 (2018). Google Scholar
- [77] Marianna Martindale and Marine Carpuat. 2018. Fluency Over Adequacy: A Pilot Study in Measuring User Trust in Imperfect MT. In Proceedings of the 13th Conference of the Association for Machine Translation in the Americas (Volume 1: Research Track). Association for Machine Translation in the Americas, Boston, MA, 13--25. https://www.aclweb.org/anthology/W18-1803 Google Scholar
- [81] Douglas M McLeod. 2007. News coverage and social protest: How the media's protect paradigm exacerbates social conflict. Journal of Dispute Resolution (2007), 185. Google Scholar
- [87] Robert C. Moore and William Lewis. 2010. Intelligent Selection of Language Model Training Data. In Proceedings of the ACL 2010 Conference Short Papers. Association for Computational Linguistics, Uppsala, Sweden, 220--224. https://www.aclweb.org/anthology/P10-2041 Google Scholar
- 10.18653/v1 · DOI
- [92] Maggie Nelson. 2015. The Argonauts. Graywolf Press, Minneapolis. Google Scholar
- 10.18653/v1 · DOI
- [96] David Ortiz, Daniel Myers, Eugene Walls, and Maria-Elena Diaz. 2005. Where do we stand with newspaper data? Mobilization: An International Quarterly 10, 3 (2005), 397--419. Google Scholar
- 10.3115/v1 · doi:10.3115/v1
- 10.18653/v1 · DOI
- 10.18653/v1 · DOI
- [106] Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language models are unsupervised multitask learners. OpenAI Blog 1, 8 (2019), 9. Google Scholar
- 10.18653/v1 · DOI
- [109] Sarah T. Roberts, Joel Tetreault, Vinodkumar Prabhakaran, and Zeerak Waseem (Eds.). 2019. Proceedings of the Third Workshop on Abusive Language Online. Association for Computational Linguistics, Florence, Italy. https://www.aclweb.org/anthology/W19-3500 Google Scholar
- [112] Corby Rosset. 2020. Turing-NLG: A 17-billion-parameter language model by Microsoft. Microsoft Blog (2020). Google Scholar
- 10.18653/v1 · DOI
- 10.18653/v1 · DOI
- [121] Joonbo Shin, Yoonhyung Lee, and Kyomin Jung. 2019. Effective Sentence Scoring Method Using BERT for Speech Recognition. In Asian Conference on Machine Learning. 1081--1093. Google Scholar
- [124] Karen Spärck Jones. 2004. Language modelling's generative model: Is it rational? Technical Report. Computer Laboratory, University of Cambridge. Google Scholar
- [127] Katrina Srigley and Lorraine Sutherland. 2019. Decolonizing, Indigenizing, and Learning Biskaaybiiyang in the Field: Our Oral History Journey1. The Oral History Review (2019). Google Scholar
- 10.18653/v1 · DOI
- 10.18653/v1 · DOI
- 10.18653/v1 · DOI
- 10.18653/v1 · DOI
- 10.18653/v1 · DOI
- 10.18653/v1 · DOI
- 10.18653/v1 · DOI
- [150] Zhilin Yang, Zihang Dai, Yiming Yang, Jaime Carbonell, Russ R Salakhutdinov, and Quoc V Le. 2019. Xlnet: Generalized autoregressive pretraining for language understanding. In Advances in neural information processing systems. 5753--5763. Google Scholar
- 10.18653/v1 · DOI
- 10.18653/v1 · DOI
- 10.18653/v1 · DOI
Cited by
Items 1,101–1,200 of 6,930
Jones, G. M., Satran, S., & Satyanarayan, A. (2024) · Big Data & Society · doi:10.1177/20539517241303118 · Cited by 23
“While publics have greeted newly loquacious chatbots with a widespread sense of awe (Jones et al., 2023; Keane and Shapiro, 2023), critical AI scholars have right-fully warned of the overblown and potentially harmful hype surrounding LLMs (Bender et al., 2021; Suchman, 2023).”
de Ligt, K. M., Hommes, S., Vromans, R., Boomstra, E., van de Poll-Franse, L. V., & Krahmer, E. (2025) · Journal of Medical Internet Research · doi:10.2196/60777 · Cited by 29
Ishmuradova, I. I., Zhdanov, S. P., Kondrashev, S. V., Erokhova, N. S., Grishnova, E. E., & Volosova, N. Y. (2025) · Contemporary Educational Technology · doi:10.30935/cedtech/16207 · Cited by 38
Aslam, M. M., Tufail, A., Gul, H., Irshad, M. N., & Namoun, A. (2025) · Artificial Intelligence Review · doi:10.1007/s10462-025-11320-9 · Cited by 41
New sources of inaccuracy? A conceptual framework for studying AI hallucinations
Shao, A. (2025) · Harvard Kennedy School Misinformation Review · doi:10.37016/mr-2020-182 · Cited by 41
“The internal operations of LLMs remain opaque or “black boxes” (Bender et al., 2021; Weidinger et al., 2022).”
Monroe‐White, T. (2021) · ACM SIGCPR Conference on Computer Personnel Research (SIGCPR) · doi:10.1145/3458026.3462161 · Cited by 24
“[61 , 62 ] What protections are in place to protect critical technologists and data scientists?”
BackgroundLeveraging Adversarial Examples to Quantify Membership Information Leakage
Grosso, G. D., Jalalzai, H., Pichler, G., Palamidessi, C., & Piantanida, P. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01015 · Cited by 42
“other societal aspects such as fairness [38, 57] or safety [5] should not be trampled on [4].”
BackgroundPatterns of Bias: How Mainstream Media Operationalize Links between Mass Shootings and Terrorism
Dreier, S. K., Gade, E. K., Card, D., & Smith, N. A. (2022) · Political Communication · doi:10.1080/10584609.2022.2111484 · Cited by 23
Jones, M., Neumayer, C., & Shklovski, I. (2023) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3580885 · Cited by 31
Understanding computational dialogue understanding
Wahlster, W. (2023) · Philosophical Transactions of the Royal Society A Mathematical Physical and Engineering Sciences · doi:10.1098/rsta.2022.0049 · Cited by 30
Neural production networks: AI’s infrastructural geographies
Ferrari, F. (2023) · Environment and Planning F · doi:10.1177/26349825231193226 · Cited by 37
Scientific Article Writing: Will ChatGPT Help?
Zashikhina, I. M. (2023) · Высшее образование в России · doi:10.31992/0869-3617-2023-32-8-9-24-47 · Cited by 30
Encountering Artificial Intelligence: Ethical and Anthropological Investigations
Gaudet, M. J., Herzfeld, N., Scherz, P., & Wales, J. J. (2023) · Journal of Moral Theology · doi:10.55476/001c.91230 · Cited by 27
KnowledgeVIS: Interpreting Language Models by Comparing Fill-in-the-Blank Prompts
Coscia, A. J., & Endert, A. (2023) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2023.3346713 · Cited by 27
“Yet automated auditing systems requiring social categories and quantitative metrics to be predetermined are prone to missing underrepresented stereotypes [12].”
BackgroundBreaking the Bias: Gender Fairness in LLMs Using Prompt Engineering and In-Context Learning
HSS, I. B., Dwivedi, S., Ghosh, S. T., & Dwivedi, S. (2023) · Rupkatha Journal on Interdisciplinary Studies in Humanities · doi:10.21659/rupkatha.v15n4.10 · Cited by 59
“Bender et al.'s argument that the focus shouldn't just be on "de-biasing" but understanding broader implications rings true in light of our findings (Bender et al., 2021).”
BackgroundResultTeBlunthuis, N., Hase, V., & Chan, C. (2024) · Communication Methods and Measures · doi:10.1080/19312458.2023.2293713 · Cited by 50
Debating AI in Archaeology: applications, implications, and ethical considerations
Tenzer, M., Pistilli, G., Bransden, A., & Shenfield, A. (2024) · Internet Archaeology · doi:10.11141/ia.67.8 · Cited by 33
Creative Dialogue with Generative AI: Exploring the Possible with Ron Beghetto
Mishra, P., & Henriksen, D. (2024) · TechTrends · doi:10.1007/s11528-024-00949-y · Cited by 27
“Although he is aware that LLMs are essentially nothing more than a “next word predictor” (a “stochastic parrot” as described by Bender, et. al, 2021), still, something surprising has emerged.”
Lim, B., Seth, I., Xie, Y., Kenney, P. S., Cuomo, R., & Rozen, W. M. (2024) · Aesthetic Plastic Surgery · doi:10.1007/s00266-024-03952-z · Cited by 24
Large Language Models and the Wisdom of Small Crowds
Trott, S. (2024) · Open Mind · doi:10.1162/opmi_a_00144 · Cited by 40
“…human participants (Henrich et al., 2010); recent work suggests that contemporary LLMs like GPT-4 are “ WEIRD ” as well (Atari et al., 2023), presumably because they are trained on data produced by an unrepresentative sample of language users (Bender et al., 2021; Crockett & Messeri, 2023).”
BackgroundA Survey of LLM Datasets: From Autoregressive Model to AI Chatbot
Du, F., Ma, X.-J., Yang, J., Liu, Y., Luo, C.-R., Wang, X., Jiang, H.-O., & Jing, X. (2024) · Journal of Computer Science and Technology · doi:10.1007/s11390-024-3767-3 · Cited by 31
Jain, R. K., & Jain, A. (2024) · Lecture notes in networks and systems · doi:10.1007/978-3-031-66329-1_42 · Cited by 55
Distortions in Judged Spatial Relations in Large Language Models
Fulman, N., Memduhoğlu, A., & Zipf, A. (2024) · The Professional Geographer · doi:10.1080/00330124.2024.2372792 · Cited by 24
“As Bender et al. (2021) posited, LLMs can be seen as stochastic parrots, merely echoing the patterns found within their vast training data sets without genuine understanding, and precise logic should not be expected of them.”
Kumar, P. C., Cotter, K., & Cabrera, L. Y. (2024) · Reading Research Quarterly · doi:10.1002/rrq.570 · Cited by 27
Shrinking Bigfoot: Reducing wav2vec 2.0 footprint
Peng, Z., Budhkar, A., Tuil, I., Levy, J., Sobhani, P., Cohen, R., & Nassour, J. (2021) · Workshop on Simple and Efficient Natural Language Processing (SustaiNLP) · doi:10.18653/v1/2021.sustainlp-1.14 · Cited by 41
Nee, J., Smith, G. M., Sheares, A. M., & Rustagi, I. (2021) · ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization (EAAMO) · doi:10.1145/3465416.3483301 · Cited by 24
“curation of datasets and monitoring of NLP tools could help shift the narrative and amplify historically marginalized perspectives [17].”
BackgroundInfluentialEthics of Conversational User Interfaces
Lee, M., Sin, J., Laban, G., Kraus, M., Clark, L., Porcheron, M., Cowan, B. R., Følstad, A., Munteanu, C., & Candello, H. C. D. S. P. (2022) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3491101.3503699 · Cited by 25
“In addition, initial summaries of ethical and societal implications have been provided for conversational AI [45] and large language models [1].”
BackgroundWhy we need biased AI: How including cognitive biases can enhance AI systems
Hagendorff, T., & Fabi, S. (2024) · Journal of Experimental & Theoretical Artificial Intelligence · doi:10.1080/0952813x.2023.2178517 · Cited by 20
“Biases are a “natural” part of large text corpora and hence large language models (Bender et al. 2021).”
Han, D., Zhao, W., Yin, H., Qu, M., Zhu, J., Ma, F., Ying, Y., & Pan, A. (2025) · Journal of Asian Architecture and Building Engineering · doi:10.1080/13467581.2024.2329351 · Cited by 20
“With the rise of deep learning (Goodfellow, Bengio, and Courville 2016; LeCun, Bengio, and Hinton 2015), recent Large Language Models (LLMs) (Bender et al. 2021; Chen et al. 2023) like Chat-GPT (n OpenAI 2023) represent remarkable advancements in the field of natural language processing (Chowdhary…”
Opportunities and Risks of LLMs for Scalable Deliberation with Polis
Small, C. T., Vendrov, I., Durmus, E., Homaei, H., Barry, E., Cornebise, J., Suzman, T., Ganguli, D., & Megill, C. (2023) · arXiv · doi:10.48550/arxiv.2306.11932 · Cited by 20
“Furthermore, LLMs may reflect and amplify societal biases present in their training data (Bender et al., 2021a; Abid et al., 2021a; Cheng et al., 2023) – including political biases – which may affect what information they extract and emphasize when summarizing deliberative discourse (see Section…”
BackgroundChatGPT and the stochastic parrot: artificial intelligence in medical research
Boussen, S., Denis, J.-B., Simeone, P., Lagier, D., Bruder, N., & Velly, L. (2023) · British Journal of Anaesthesia · doi:10.1016/j.bja.2023.06.065 · Cited by 20
Signs of consciousness in AI: Can GPT-3 tell how smart it really is?
Bojić, L., Stojković, I., & Jolić Marjanović, Z. (2024) · Humanities and Social Sciences Communications · doi:10.1057/s41599-024-04154-3 · Cited by 19
“As Bender et al. (2021) aptly noted, these models may best be described as “ stochastic parrots, ” with their apparent coherence often mistaken for consciousness.”
A Copious Void: Rhetoric as Artificial Intelligence 1.0
Hallsby, A. (2024) · Rhetoric Society Quarterly · doi:10.1080/02773945.2024.2343265 · Cited by 23
Thominet, L., Amorim, J., Acosta, K., & Sohan, V. K. (2024) · Journal of Technical Writing and Communication · doi:10.1177/00472816241260044 · Cited by 24
“…already warned against anthropomorphizing LLM chat-bots, which primarily seek to please users (Wei et al., 2024), con fi dently state false-hoods (Roberts et al., 2024), are trained on a biased dataset (Algorithmic Justice League, 2024), and lack a mind to make meaning with (Bender et al., 2021).”
Hamid, O. H. (2024) · IEEE Conference on Cognitive and Computational Aspects of Situation Management (CogSIMA) · doi:10.1109/cogsima61085.2024.10553755 · Cited by 35
“However, it resembles a statistical understanding rather than a grammatical one [12].”
BackgroundThe rise of large language models: challenges for Critical Discourse Studies
Gillings, M., Kohn, T., & Mautner, G. (2024) · Critical Discourse Studies · doi:10.1080/17405904.2024.2373733 · Cited by 50
“Thus, existing biases and discrimination may be amplified, and hegemonic views reinforced (Jungherr, 2023, p. 7; Bender et al., 2021, p. 613).”
BackgroundHe, L., Başar, E., Krahmer, E., Wiers, R. W. H. J., & Antheunis, M. L. (2024) · Journal of Medical Internet Research · doi:10.2196/53134 · Cited by 39
Do large language models “understand” their knowledge?
Venkatasubramanian, V. (2024) · AIChE Journal · doi:10.1002/aic.18661 · Cited by 24
Chauhan, A., Khaliq, F., & Nayak, K. R. (2025) · International Journal of Artificial Intelligence in Education · doi:10.1007/s40593-025-00471-z · Cited by 32
“This bias arises from ChatGPT’s training data, which is predominantly sourced from global texts that often reflect Western English norms, neglecting regional linguistic nuances(Bender et al., 2021).”
Strengths and Weaknesses of LLM-Based and Rule-Based NLP Technologies and Their Potential Synergies
Karanikolas, N. Ν., Manga, E., Samaridi, N. E., Stergiopoulos, V., Tousidou, E., & Vassilakopoulos, M. G. (2025) · Electronics · doi:10.3390/electronics14153064 · Cited by 37
Deng, C., Duan, Y., Jin, X., Chang, H., Tian, Y., Liu, H., Wang, Y., Gao, K., Zou, H. P., Jin, Y., et al. (2025) · AI and Ethics · doi:10.1007/s43681-025-00797-3 · Cited by 85
“LLMs inherit and potentially amplify societal biases pres-ent in their training data, which can perpetuate harm against marginalized communities [21].”
Company Analysis of OpenAI with Special Emphasis on its Future Strategies
Yashwi, & Aithal, P. S. (2025) · Poornaprajna International Journal of Emerging Technologies (PIJET) · doi:10.64818/pijet.3107.8486.0009 · Cited by 34
Long-Tail Zero and Few-Shot Learning via Contrastive Pretraining on and for Small Data
Rethmeier, N., & Augenstein, I. (2022) · doi:10.3390/cmsf2022003010 · Cited by 23
Carbon-Responsive Computing: Changing the Nexus between Energy and Computing
Nafus, D., Schooler, E. M., & Burch, K. A. (2021) · Energies · doi:10.3390/en14216917 · Cited by 29
“There is a good deal of research on energy consumption of individual devices (see [47] for edge devices), and there is growing interest in measuring the energy consumption of AI models both as a matter of academic research [68,69] and in benchmarking [70].”
BackgroundCurrent Advances in Neural Networks
Gallego, V., & Insua, D. R. (2021) · Annual Review of Statistics and Its Application · doi:10.1146/annurev-statistics-040220-112019 · Cited by 34
Willem, T., Krammer, S., Böhm, A. ‐., French, L. E., Hartmann, D., Lasser, T., & Buyx, A. (2022) · Journal of the European Academy of Dermatology and Venereology · doi:10.1111/jdv.18192 · Cited by 23
“46,47 This could apply particularly to those people who work in the mining of rare earth materials that are needed for machine learning health care applications and suffer health consequences from their work; those who are too poor to gain access to, or are educationally unaware of the function and…”
BackgroundCâmara, A., Taneja, N., Azad, T., Allaway, E., & Zemel, R. S. (2022) · Workshop on Language Technology for Equality, Diversity, Inclusion (LT-EDI) · doi:10.18653/v1/2022.ltedi-1.11 · Cited by 35
Practicing Humility: Design as Response, Not as Solution
Spiel, K. (2023) · Postdigital Science and Education · doi:10.1007/s42438-023-00436-2 · Cited by 24
“…in the context of technological design and increased efforts of digitising our lived environments and socio-technical constraints, however, we have learned that these ‘solutions’ often lead to more widespread and manifest issues around discrimination (Bender et al. 2021) and exclusion (Spiel 2021).”
BackgroundAI ethics and ordoliberalism 2.0: towards a ‘Digital Bill of Rights’
Wörsdörfer, M. (2023) · AI and Ethics · doi:10.1007/s43681-023-00367-5 · Cited by 22
Evaluation of African American Language Bias in Natural Language Generation
Deas, N., Grieser, J. A., Kleiner, S., Patton, D. U., Turcan, E., & McKeown, K. R. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.421 · Cited by 92
AI meets comedy: Viewers' reactions to GPT-4 generated humor translation
Abu-Rayyash, H. (2023) · Ampersand · doi:10.1016/j.amper.2023.100162 · Cited by 27
Kotturi, Y., Anderson, A., Ford, G. J., Skirpan, M. W., & Bigham, J. P. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642191 · Cited by 31
Easy-read and large language models: on the ethical dimensions of LLM-based text simplification
Freyer, N., Kempt, H., & Klöser, L. (2024) · Ethics and Information Technology · doi:10.1007/s10676-024-09792-4 · Cited by 38
“Namely, the mitigation of biases and the ensuring of appropriate and considerably safe text outputs (Bender et al., 2021; Floridi, 2023; Kempt et al., 2023).”
Stochastic lies: How LLM-powered chatbots deal with Russian disinformation about the war in Ukraine
Makhortykh, M., Sydorova, M., Baghumyan, A., Vziatysheva, V., & Kuznetsova, E. (2024) · Harvard Kennedy School Misinformation Review · doi:10.37016/mr-2020-154 · Cited by 19
“To produce new content, chatbots take user prompts as input and predict the most likely sequence of linguistic tokens (e.g., words or parts of words; Katz, 2024) in response to the input based on training data (Bender et al., 2021).”
BackgroundHumans vs. large language models: Judgmental forecasting in an era of advanced AI
Abolghasemi, M., Ganbold, O., & Rotaru, K. (2024) · International Journal of Forecasting · doi:10.1016/j.ijforecast.2024.07.003 · Cited by 58
Machine Bias. How Do Generative Language Models Answer Opinion Polls?
Boelaert, J., Coavoux, S., Ollion, É., Petev, I. D., & Präg, P. (2025) · Sociological Methods & Research · doi:10.1177/00491241251330582 · Cited by 88
“Just like widely used predictive tools based on statistical learning (Buolamwini and Gebru 2018), LLMs could harbor massive social biases (Bender et al. 2021).”
Ethical and Responsible Use of AI/ML in the Earth, Space, and Environmental SciencesPreprint
Stall, S., Cervone, G., Coward, C., Cutcher‐Gershenfeld, J., Donaldson, T. J., Erdmann, C. C., Hanson, B., Holm, J., King, J. L., Lyon, L., et al. (2023) · doi:10.22541/essoar.168132856.66485758/v1 · Cited by 28
AI is Like… A Literature Review of AI Metaphors and Why They Matter for PolicyPreprint
Maas, M. M. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4612468 · Cited by 33
Hung, K. (2024) · Big Data & Society · doi:10.1177/20539517241289443 · Cited by 24
“It emphasizes the skewed political economy of knowledge production, building out technical divisions of knowledge, expertise, and skills that privilege dominant data epistemologies as exempli fi ed by connectionist forms of AI (see Bender et al., 2021; LeCun et al., 2015).”
Evidence for Hypodescent in Visual Semantic AI
Wolfe, R., Banaji, M. R., & Caliskan, A. (2022) · 2022 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3531146.3533185 · Cited by 19
Toward a Theory of Causation for Interpreting Neural Code Models
Nader Palacio, D., Velasco, A., Cooper, N., Rodriguez, A., Moran, K., & Poshyvanyk, D. (2024) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2024.3379943 · Cited by 18
“Restrictions apply. and thus may inherit the various limitations often associated with such models — including biases, memorization, and issues with data inefficiency, to name a few [22].”
BackgroundChen, M. (2023) · AI & Society · doi:10.1007/s00146-023-01681-6 · Cited by 27
“Nonetheless and as things stand, we are still dealing with stochastic parrots or lumbering statistical machines for pattern-matching, rather than entities that are capable of natural language understanding after the manner of human beings (Bender et al. 2021; Chomsky et al. 2023).”
Talking existential risk into being: a Habermasian critical discourse perspective to AI hype
Westerstrand, S., Westerstrand, R., & Koskinen, J. S. S. (2024) · AI and Ethics · doi:10.1007/s43681-024-00464-z · Cited by 31
Volk, S. C., Schäfer, M. S., Lombardi, D., Mahl, D., & Yan, X. (2024) · Public Understanding of Science · doi:10.1177/09636625241268910 · Cited by 22
Artificial Intelligence (AI) and Global Justice
Sahebi, S., & Formosa, P. (2024) · Minds and Machines · doi:10.1007/s11023-024-09708-7 · Cited by 28
“GPUs and other core components for computing require large amounts of minerals and resources (Crawford, 2021 ), as well as power to run them (McQuillan, 2022; Bender et al., 2021 ).”
Standards for Belief Representations in LLMs
Herrmann, D. A., & Levinstein, B. A. (2024) · Minds and Machines · doi:10.1007/s11023-024-09709-6 · Cited by 84
“Some have expressed skepticism that LLMs have anything resembling beliefs, even in principle (Bender & Koller, 2020; Bender et al., 2021; Shanahan, 2022).”
Rojas, Ó. A. C., Contero, M., & Hincapié, M. (2025) · Multimodal Technologies and Interaction · doi:10.3390/mti9050045 · Cited by 41
Narayanan Venkit, P., Laban, P., Zhou, Y., Mao, Y., & Wu, C.-S. (2025) · Proceedings of the 2025 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3715275.3732089 · Cited by 23
“Another key concern surrounding answer engines is their inherently ‘black-box’ nature [3, 50] due to the opacity of their decision-making processes and the hidden biases within them [3, 28, 41].”
How Good Is ChatGPT in Giving Advice on Your Visualization Design?
Kim, N. W., Ahn, Y., Myers, G., & Bach, B. (2025) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3745768 · Cited by 55
“[1] These professionals do not typically have formal training or education in data visualization, while mostly learning necessary visualization skills on the go [1], [2].”
MoralBench: Moral Evaluation of LLMs
Ji, J., Chen, Y., Jin, M., Xu, W., Hua, W., & Zhang, Y. (2025) · ACM SIGKDD Explorations Newsletter · doi:10.1145/3748239.3748246 · Cited by 150
“…with LLMs stemming from the voluminous, yet noisy, training datasets; these can lead to the inadvertent generation of biased or harmful content, such as gender and religious prejudices as well as aggressive language [52; 53; 54; 55; 56], thereby undermining their reliability and trustworthiness.”
Securing the Imagination: Security and Privacy Challenges in Generative AI Systems
Goyal, N. K., Gulati, K., Avasthi, A., Kapoor, S. K., Keswani, B., & Guptā, S. (2025) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-3373-5616-7.ch010 · Cited by 18
A Cognitive Regularizer for Language Modeling
Wei, J., Meister, C., & Cotterell, R. (2021) · doi:10.18653/v1/2021.acl-long.404 · Cited by 31
“We do not foresee any additional ethical concerns with the contributions made in our work beyond those discussed in Bender et al. (2021).”
BackgroundTrust Me on This One: Conforming to Conversational Assistants
Schreuter, D., van der Putten, P., & Lamers, M. H. (2021) · Minds and Machines · doi:10.1007/s11023-021-09581-8 · Cited by 22
DiLBERT: Cheap Embeddings for Disease Related Medical NLP
Roitero, K., Portelli, B., Popescu, M. H., & Mea, V. D. (2021) · IEEE Access · doi:10.1109/access.2021.3131386 · Cited by 25
“We like to remark that the usage of less resources, and in particular a significantly smaller training set is a direction that many researchers are trying to address in the case of deep learning models (see for example [11]).”
BackgroundThe AI trilemma: Saving the planet without ruining our jobs
Ernst, E. (2022) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2022.886561 · Cited by 29
“Related, narrowing AI research and rising ecological and economic cost lead to market concentration, both in the development and training of new (large) models (Bender et al., 2021) and in related digital applications such as blockchain applications in cryptocurrency markets, where similar tendencies to oligopolistic…”
BackgroundPotential Pitfalls With Automatic Sentiment Analysis: The Example of Queerphobic Bias
Ungless, E. L., Roß, B., & Belle, V. (2023) · Social Science Computer Review · doi:10.1177/08944393231152946 · Cited by 24
“This contrasts with the ‘ black box ’ nature of modern deep learning approaches where carefully constructed probes are needed to ‘ tease out ’ patterns of bias (Bender et al., 2021), and addressing this bias involves more complex processes (Meade et al., 2021).”
BackgroundThe position of machine translation in translation studies: A definitional perspective
Asscher, O. (2023) · Translation Spaces · doi:10.1075/ts.22035.ass · Cited by 31
Broadbent, M., Grespan, M. M., Axford, K., Zhang, X., Srikumar, V., Kious, B. M., & Imel, Z. E. (2023) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2023.1110527 · Cited by 36
Ten years after ImageNet: a 360° perspective on artificial intelligence
Chawla, S., Nakov, P., Ali, A., Hall, W., Khalil, I. M., Ma, X., Sencar, H. T., Weber, I. G., Wooldridge, M., & Yu, T. (2023) · Royal Society Open Science · doi:10.1098/rsos.221414 · Cited by 28
“[47], who coined the phrase ‘stochastic parrots’ to describe large LMs.”
BackgroundFraming the News: From Human Perception to Large Language Model Inferences
del Barrio, D. A., & Gática-Pérez, D. (2023) · International Conference on Multimedia Retrieval · doi:10.1145/3591106.3592278 · Cited by 39
The Edinburgh International Accents of English Corpus: Towards the Democratization of English ASR
Sanabria, R., Bogoychev, N., Markl, N., Carmantini, A., Klejch, O., & Bell, P. (2023) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49357.2023.10095057 · Cited by 66
“Language change, just like language variation, is an inherent and natural feature of language, and therefore older datasets further risk becoming unrepresentative of current language use – a problem relevant to all fields of natural language processing, eg., [13].”
BackgroundOn The Impact of Machine Learning Randomness on Group Fairness
Ganesh, P., Chang, H., Strobel, M., & Shokri, R. (2023) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3593013.3594116 · Cited by 63
Krüger, R. (2023) · Lebende Sprachen · doi:10.1515/les-2023-0024 · Cited by 37
Navigating Ethical Complexities Through Epistemological Analysis of ChatGPT
Chavanayarn, S. (2023) · Bulletin of Science Technology & Society · doi:10.1177/02704676231216355 · Cited by 26
“The system exhibited a tendency to associate certain occupations and roles with speci fi c genders, perpetuating gender stereotypes prevalent in the training data (Bender & Gebru, 2021).”
BackgroundMoT: Memory-of-Thought Enables ChatGPT to Self-Improve
Li, X., & Qiu, X. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.392 · Cited by 104
“On the other hand, large language models have shown various kinds of bias (Bender et al., 2021).”
BackgroundThe Sentiment Problem: A Critical Survey towards Deconstructing Sentiment Analysis
Venkit, P. N., Srinath, M., Gautam, S., Venkatraman, S., Gupta, V. K., Passonneau, R. J., & Wilson, S. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.848 · Cited by 41
Nishal, S., Sinchai, J., & Diakopoulos, N. A. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3637419 · Cited by 33
“Journalists also desired transparency about training data and models, especially to reveal fallacies in generative models [10, 56].”
BackgroundMethodConstantinides, M., Tahaei, M., Quercia, D., Stumpf, S., Madaio, M., Kennedy, S., Wilcox, L. G., Vitak, J., Cramer, H. S. M., Bogucka, E. P., et al. (2024) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3613905.3643979 · Cited by 32
How to Regulate Large Language Models for Responsible AI
Berengueres, J. O. L. (2024) · IEEE Transactions on Technology and Society · doi:10.1109/tts.2024.3403681 · Cited by 33
“A non-exhaustive list of such research follows: environmental impact [28], [29], certi-fication [30], trustworthiness [31], AI colonialism [32], data nationalism [33], bias [34], future of work [35], and domain-specific applications such as healthcare.”
BackgroundA Review of State of the Art Deep Learning Models for Ontology Construction
Zengeya, T., & Fonou-Dombeu, J. V. (2024) · IEEE Access · doi:10.1109/access.2024.3406426 · Cited by 33
“However, there were concerns about generating inappropriate or biased content by these Chat-GPT models [52], [53].”
BackgroundCollage is the New Writing: Exploring the Fragmentation of Text and User Interfaces in AI Tools
Buschek, D. (2024) · Designing Interactive Systems Conference · doi:10.1145/3643834.3660681 · Cited by 44
“Indeed, related work found that LLMs have biases [9], favour some views over others [46], and – through interaction – may influence both writing and writers [44].”
BackgroundAuditing GPT's Content Moderation Guardrails: Can ChatGPT Write Your Favorite TV Show?
Mahomed, Y., Crawford, C. M., Gautam, S., Friedler, S. A., & Metaxa, D. (2024) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3630106.3658932 · Cited by 38
“The full set of tags associated with each tag group is given in Appendix Tables 7, 6, 8, and 9.”
BackgroundLUNA: A Model-Based Universal Analysis Framework for Large Language Models
Song, D., Xie, X., Song, J., Zhu, D., Huang, Y., Juefei-Xu, F., & Ma, L. (2024) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2024.3411928 · Cited by 41
“As introduced in Section II, abnormal behavior refers to the unintended expression of LLM, e.g., making up answers or generating biased output [149], [150], [151], [152], [153].”
BackgroundSupporting Trustworthy AI Through Machine Unlearning
Hine, E., Novelli, C., Taddeo, M., & Floridi, L. (2024) · Science and Engineering Ethics · doi:10.1007/s11948-024-00500-5 · Cited by 40
Is Artificial Intelligence the Future of Collective Memory?
Gensburger, S., & Clavert, F. (2024) · Memory studies review. · doi:10.1163/29498902-202400019 · Cited by 28
Guiding Principles for Participatory Design-inspired Natural Language Processing
Caselli, T., Cibin, R., Conforti, C., Encinas, E., & Teli, M. (2021) · Workshop on NLP for Positive Impact (NLP4PI) · doi:10.18653/v1/2021.nlp4posimpact-1.4 · Cited by 51
Aligning to Social Norms and Values in Interactive Narratives
Ammanabrolu, P., Jiang, L., Sap, M., Hajishirzi, H., & Choi, Y. (2022) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2022.naacl-main.439 · Cited by 54
What Do NLP Researchers Believe? Results of the NLP Community Metasurvey
Michael, J., Holtzman, A., Parrish, A., Mueller, A., Wang, A., Chen, A., Madaan, D., Nangia, N., Pang, R. Y., Phang, J., et al. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.903 · Cited by 65
Prettyman, A. (2026) · Inquiry · doi:10.1080/0020174x.2024.2439989 · Cited by 18
“They and their colleagues allege that LLMs carry signi fi cant risks for human society, some of which have already come to pass (Bender et al. 2021).”
InfluentialOligarchy, State, and CryptopiaPreprint
Cohen, J. E. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5171050 · Cited by 28
Metadata
- Authors
- Bender, E. M., Gebru, T., McMillan-Major, A., & Shmitchell, S.
- Affiliations
- University of Washington (US)
- Black in AI, Palo Alto, CA, USA
- The Aether
- Venue
- Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency
- Publisher
- ACM
- Published
- 2021-03
- Pages
- 610-623
- OpenAlex
- W3133702157
- Cited by
- 6,988
- References
- 117
- Citations by year
- 2026: 2484 · 2025: 1887 · 2024: 1153 · 2023: 842 · 2022: 384 · 2021: 229
- FWCI
- 300.72
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- ACM Conference on Fairness, Accountability, and Transparency (FAccT) (publishedVersion, cc-by)
- Creighton Digital Repository (CDR) (submittedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Keywords
- large language models, model size, AI risks, financial costs, data curation, dataset documentation, stakeholder value, BERT, GPT, fine-tuning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.