On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜
Open accessThe DOI matches its Crossref record; 156 of the 159 references it cites were found in a registry or an index, and 3 are not checked yet.
Abstract
The past 3 years of work in NLP have been characterized by the development and deployment of ever larger language models, especially for English. BERT, its variants, GPT-2/3, and others, most recently Switch-C, have pushed the boundaries of the possible both through architectural innovations and through sheer size. Using these pretrained models and the methodology of fine-tuning them for specific tasks, researchers have extended the state of the art on a wide array of tasks as measured by leaderboards on specific benchmarks for English. In this paper, we take a step back and ask: How big is too big? What are the possible risks associated with this technology and what paths are available for mitigating those risks? We provide recommendations including weighing the environmental and financial costs first, investing resources into curating and carefully documenting datasets rather than ingesting everything on the web, carrying out pre-development exercises evaluating how the planned approach fits into research and development goals and supports stakeholder values, and encouraging research directions beyond ever larger language models.
Checked 2026-11-22 · 3 references not checked yet
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The text we found for this paper looks like another document, so the self-check wasn’t run.
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325 · cited 5×
“[39] in particular noted that training on a large dataset and fine-tuning for specific tasks leads to strictly increasing results on the GLUE tasks [138] for English as the hyperparameters of the model were increased.”
BackgroundMethodResultInfluentialGlove: Global Vectors for Word Representation
Pennington, J., Socher, R., & Manning, C. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1162 · Cited by 34,071
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372 · cited 4×
“documents linked to from Reddit [25], plus Wikipedia and a collection”
BackgroundMethodInfluentialRoBERTa: A Robustly Optimized BERT Pretraining Approach
Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., & Stoyanov, V. (2019) · arXiv · doi:10.48550/arxiv.1907.11692 · Cited by 16,515
Distilling the Knowledge in a Neural Network
Hinton, G., Vinyals, O., & Dean, J. (2015) · arXiv · doi:10.48550/arxiv.1503.02531 · Cited by 14,204 · cited 1×
“[26, 58], quantization [118, 153], factorized embedding parameterization and cross-layer parameter sharing [70], and progressive module replacing [146].”
Deep Contextualized Word Representations
Peters, M., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., & Zettlemoyer, L. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1202 · Cited by 11,983
Proceedings of the 24th international conference on Machine learning
(2007) · doi:10.1145/1273496 · Cited by 11,630
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., & Liu, P. J. (2019) · arXiv · doi:10.48550/arxiv.1910.10683 · Cited by 10,929
The Mathematical Theory of Communication
Shannon, C. E., Weaver, W., & Wiener, N. (1950) · Physics Today · doi:10.1063/1.3067010 · Cited by 9,801
Research works printed but not linked: 48
- [4] American Psychological Association. 2019. Discrimination: What it is, and how to cope. https://www.apa.org/topics/discrimination (2019). Google Scholar
- 10.18653/v1 · DOI
- 10.18653/v1 · DOI
- [15] Ruha Benjamin. 2019. Race After Technology: Abolitionist Tools for the New Jim Code. Polity Press, Cambridge, UK. Crossref Google Scholar
- [16] Elettra Bietti and Roxana Vatanparast. 2020. Data Waste. Harvard International Law Journal 61 (2020). Google Scholar
- 10.18653/v1 · DOI
- 10.18653/v1 · DOI
- 10.1016/0010-0277(86) · doi:10.1016/0010-0277(86
- [38] Terrance de Vries, Ishan Misra, Changhan Wang, and Laurens van der Maaten. 2019. Does object recognition work for everyone?. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. 52--59. Google Scholar
- 10.18653/v1 · DOI
- [42] Ethan Fast, Tina Vachovsky, and Michael Bernstein. 2016. Shirtless and Dangerous: Quantifying Linguistic Signals of Gender Bias in an Online Fiction Writing Community. In Proceedings of the International AAAI Conference on Web and Social Media, Vol. 10. Google Scholar
- 10.18653/v1 · DOI
- [45] Darja Fišer, Ruihong Huang, Vinodkumar Prabhakaran, Rob Voigt, Zeerak Waseem, and Jacqueline Wernimont (Eds.). 2018. Proceedings of the 2nd Workshop on Abusive Language Online (ALW2). Association for Computational Linguistics, Brussels, Belgium. https://www.aclweb.org/anthology/W18-5100 Google Scholar
- [50] Batya Friedman, Peter H. Kahn, Jr., and Alan Borning. 2006. Value sensitive design and information systems. In Human-Computer Interaction in Management Information Systems: Foundations, P Zhang and D Galletta (Eds.). M. E. Sharpe, Armonk NY, 348--372. Google Scholar
- 10.18653/v1 · DOI
- 10.18653/v1 · DOI
- 10.18653/v1 · DOI
- [67] Brendan Kennedy, Drew Kogon, Kris Coombs, Joseph Hoover, Christina Park, Gwenyth Portillo-Wightman, Aida Mostafazadeh Davani, Mohammad Atari, and Morteza Dehghani. 2018. A typology and coding manual for the study of hate-based rhetoric. PsyArXiv. July 18 (2018). Google Scholar
- [77] Marianna Martindale and Marine Carpuat. 2018. Fluency Over Adequacy: A Pilot Study in Measuring User Trust in Imperfect MT. In Proceedings of the 13th Conference of the Association for Machine Translation in the Americas (Volume 1: Research Track). Association for Machine Translation in the Americas, Boston, MA, 13--25. https://www.aclweb.org/anthology/W18-1803 Google Scholar
- [81] Douglas M McLeod. 2007. News coverage and social protest: How the media's protect paradigm exacerbates social conflict. Journal of Dispute Resolution (2007), 185. Google Scholar
- [87] Robert C. Moore and William Lewis. 2010. Intelligent Selection of Language Model Training Data. In Proceedings of the ACL 2010 Conference Short Papers. Association for Computational Linguistics, Uppsala, Sweden, 220--224. https://www.aclweb.org/anthology/P10-2041 Google Scholar
- 10.18653/v1 · DOI
- [92] Maggie Nelson. 2015. The Argonauts. Graywolf Press, Minneapolis. Google Scholar
- 10.18653/v1 · DOI
- [96] David Ortiz, Daniel Myers, Eugene Walls, and Maria-Elena Diaz. 2005. Where do we stand with newspaper data? Mobilization: An International Quarterly 10, 3 (2005), 397--419. Google Scholar
- 10.3115/v1 · doi:10.3115/v1
- 10.18653/v1 · DOI
- 10.18653/v1 · DOI
- [106] Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language models are unsupervised multitask learners. OpenAI Blog 1, 8 (2019), 9. Google Scholar
- 10.18653/v1 · DOI
- [109] Sarah T. Roberts, Joel Tetreault, Vinodkumar Prabhakaran, and Zeerak Waseem (Eds.). 2019. Proceedings of the Third Workshop on Abusive Language Online. Association for Computational Linguistics, Florence, Italy. https://www.aclweb.org/anthology/W19-3500 Google Scholar
- [112] Corby Rosset. 2020. Turing-NLG: A 17-billion-parameter language model by Microsoft. Microsoft Blog (2020). Google Scholar
- 10.18653/v1 · DOI
- 10.18653/v1 · DOI
- [121] Joonbo Shin, Yoonhyung Lee, and Kyomin Jung. 2019. Effective Sentence Scoring Method Using BERT for Speech Recognition. In Asian Conference on Machine Learning. 1081--1093. Google Scholar
- [124] Karen Spärck Jones. 2004. Language modelling's generative model: Is it rational? Technical Report. Computer Laboratory, University of Cambridge. Google Scholar
- [127] Katrina Srigley and Lorraine Sutherland. 2019. Decolonizing, Indigenizing, and Learning Biskaaybiiyang in the Field: Our Oral History Journey1. The Oral History Review (2019). Google Scholar
- 10.18653/v1 · DOI
- 10.18653/v1 · DOI
- 10.18653/v1 · DOI
- 10.18653/v1 · DOI
- 10.18653/v1 · DOI
- 10.18653/v1 · DOI
- 10.18653/v1 · DOI
- [150] Zhilin Yang, Zihang Dai, Yiming Yang, Jaime Carbonell, Russ R Salakhutdinov, and Quoc V Le. 2019. Xlnet: Generalized autoregressive pretraining for language understanding. In Advances in neural information processing systems. 5753--5763. Google Scholar
- 10.18653/v1 · DOI
- 10.18653/v1 · DOI
- 10.18653/v1 · DOI
Cited by
Items 201–300 of 6,931
Freyer, O., Wiest, I. C., Kather, J. N., & Gilbert, S. (2024) · The Lancet Digital Health · doi:10.1016/s2589-7500(24)00124-9 · Cited by 157
Good models borrow, great models steal: intellectual property rights and generative AI
Chesterman, S. (2025) · Policy and Society · doi:10.1093/polsoc/puae006 · Cited by 156
Chan, A. (2023) · AI and Ethics · doi:10.1007/s43681-022-00148-6 · Cited by 155
"I wouldn't say offensive but...": Disability-Centered Perspectives on Large Language Models
Gadiraju, V., Kane, S., Dev, S., Taylor, A., Wang, D., Denton, R., & Brewer, R. (2023) · 2023 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3593013.3593989 · Cited by 155
Legalbench: A Collaboratively Built Benchmark for Measuring Legal Reasoning in Large Language Models
Guha, N., Nyarko, J., Ho, D. E., Ré, C., Chilton, A., Narayana, A., Chohlas-Wood, A., Peters, A., Waldon, B., Rockmore, D., et al. (2023) · doi:10.2139/ssrn.4583531 · Cited by 154
Gongane, V. U., Munot, M. V., & Anuse, A. D. (2022) · Social Network Analysis and Mining · doi:10.1007/s13278-022-00951-3 · Cited by 153
“…kind of bias with more negative sen-timents toward specific groups and overrepresentation for words like extremists, toxic and violent content (Bender et al. 2021) This kind of bias in the training characteristics of language models can show a potential risk of wrong judgment when deployed…”
BackgroundMethodInfluentialLooking Beyond the Hype: Understanding the Effects of AI on Learning
Bauer, E., Greiff, S., Graesser, A. C., Scheiter, K., & Sailer, M. (2025) · Educational Psychology Review · doi:10.1007/s10648-025-10020-8 · Cited by 153
“Instead, these models generate text by probabilistically predicting word sequences based on patterns in their training data, a process which has been compared to a “stochastic par-rot” (Bender et al., 2021).”
PromptRobust: Towards Evaluating the Robustness of Large Language Models on Adversarial Prompts
Zhu, K., Wang, J., Zhou, J., Wang, Z., Chen, H., Wang, Y., Yang, L., Ye, W., Zhang, Y., Gong, N. Z., et al. (2023) · arXiv · doi:10.48550/arxiv.2306.04528 · Cited by 153
Toxicity in chatgpt: Analyzing persona-assigned language models
Deshpande, A., Murahari, V., Rajpurohit, T., Kalyan, A., & Narasimhan, K. (2023) · Findings of the Association for Computational Linguistics: EMNLP 2023 · doi:10.18653/v1/2023.findings-emnlp.88 · Cited by 178
Theory Is All You Need: AI, Human Cognition, and Causal Reasoning
Felin, T., & Holweg, M. (2024) · doi:10.2139/ssrn.4737265 · Cited by 152
How AI can distort human beliefs
Kidd, C., & Birhane, A. (2023) · Science · doi:10.1126/science.adi0248 · Cited by 151
GPT3Mix: Leveraging Large-scale Language Models for Text Augmentation
Yoo, K. M., Park, D., Kang, J., Lee, S.-W., & Park, W. (2021) · Findings of the Association for Computational Linguistics: EMNLP 2021 · doi:10.18653/v1/2021.findings-emnlp.192 · Cited by 151
“Pre-trained language models that are trained on untreated corpora are known to exhibit social biases (Bordia and Bowman, 2019; Hutchinson et al., 2020; Abid et al., 2021; Bender et al., 2021) and toxicity (Gehman et al., 2020).”
BackgroundA Bibliometric Review of Large Language Models Research from 2017 to 2023
Fan, L., Li, L., Ma, Z., Lee, S., Yu, H., & Hemphill, L. (2024) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3664930 · Cited by 196
“For example, in question-answering systems or chatbots, models can mimic human-like thought and behavior, like “stochastic parrots”, without fully understanding the implications of such technology (Bender et al., 2021).”
(2021) · doi:10.1145/3462757.3466088 · Cited by 151
PromptRobust: Towards Evaluating the Robustness of Large Language Models on Adversarial Prompts
Zhu, K., Wang, J., Zhou, J., Wang, Z., Chen, H., Wang, Y., Yang, L., Ye, W., Zhang, Y., Gong, N., et al. (2023) · Proceedings of the 1st ACM Workshop on Large AI Systems and Models with Privacy and Safety Analysis · doi:10.1145/3689217.3690621 · Cited by 151
“Recent studies suggest that the remarkable performance of some LLMs might be rooted in their ability to memorize training data (Bender et al., 2021; Magar & Schwartz, 2022; Carlini et al., 2023; Biderman et al., 2023), rather than in generalization.”
Implicit-Bias Remedies: Treating Discriminatory Bias as a Public-Health Problem
Greenwald, A. G., Dasgupta, N., Dovidio, J. F., Kang, J., Moss‐Racusin, C. A., & Teachman, B. A. (2022) · Gothic.net · doi:10.1177/15291006211070781 · Cited by 192
“See also Payne et al. (2017, pp. 233–234).”
User Experience Design Professionals’ Perceptions of Generative Artificial Intelligence
Li, J., Cao, H., Lin, L., Hou, Y., Zhu, R., & El Ali, A. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642114 · Cited by 163
“Theme 6 provided insights into participants’ visions and desires related to the integration of GenAI in the field of design.”
BackgroundHarms from Increasingly Agentic Algorithmic Systems
Chan, A., Salganik, R., Markelius, A., Pang, C., Rajkumar, N., Krasheninnikov, D., Langosco, L., He, Z., Duan, Y., Carroll, M., et al. (2023) · 2023 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3593013.3594033 · Cited by 149
“the past, such as environmental risks [23], concentration of power”
BackgroundInfluentialFeng, S., Park, C. Y., Liu, Y., & Tsvetkov, Y. (2023) · Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2023.acl-long.656 · Cited by 149
DExperts: Decoding-Time Controlled Text Generation with Experts and Anti-Experts
Liu, A., Sap, M., Lu, X., Swayamdipta, S., Bhagavatula, C., Smith, N. A., & Choi, Y. (2021) · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.18653/v1/2021.acl-long.522 · Cited by 148
“For a broader discussion of such risks, and of the risks of large pretrained LMs in general, please see Bender et al. (2021).”
BackgroundA Literature Review of Textual Hate Speech Detection Methods and Datasets
Alkomah, F., & Ma, X. (2022) · Information · doi:10.3390/info13060273 · Cited by 256
“Therefore, the size of the dataset does not guarantee diversity [137].”
BackgroundArtificial Intelligence in the Colonial Matrix of Power
Muldoon, J., & Wu, B. A. (2023) · Philosophy & Technology · doi:10.1007/s13347-023-00687-8 · Cited by 148
“The environmental costs of this technology are not distributed equally, with countries vulnerable to climate change related catastrophes most at risk (Bender et al., 2021; Westra & Lawson, 2001).”
InfluentialBeyond Goldfish Memory: Long-Term Open-Domain Conversation
Xu, J., Szlam, A., & Weston, J. (2021) · arXiv · doi:10.48550/arxiv.2107.07567 · Cited by 148
Design and assessment of AI-based learning tools in higher education: a systematic review
Luo, J., Zheng, C., Yin, J., & Teo, H. (2025) · International Journal of Educational Technology in Higher Education · doi:10.1186/s41239-025-00540-2 · Cited by 343
“Ethical evaluations must also encompass environmental and sustainable considerations, acknowledging the substantial computational resources required to train and deploy large-scale AI models (Bender et al., 2021).”
InfluentialGenerative AI and the Automating of Academia
Watermeyer, R., Phipps, L., Lanclos, D., & Knight, C. (2024) · Postdigital Science and Education · doi:10.1007/s42438-023-00440-6 · Cited by 145
“Where LLMs are made analogous with ‘sto-chastic parrots’ (Bender et al. 2021), the multi-applicability of their use and near future proliferation into academic work demands that academics urgently confront and honestly reappraise the purpose and value of what they do lest both are sunk (cf. Welsh…”
Argyle, L. P., Bail, C. A., Busby, E. C., Gubler, J. R., Howe, T., Rytting, C., Sorensen, T., & Wingate, D. (2023) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2311627120 · Cited by 144
Measuring algorithmically infused societies
Wagner, C., Strohmaier, M., Olteanu, A., Kıcıman, E., Contractor, N., & Eliassi-Rad, T. (2021) · Nature · doi:10.1038/s41586-021-03666-1 · Cited by 144
When Human-AI Interactions Become Parasocial: Agency and Anthropomorphism in Affective Design
Maeda, T., & Quan-Haase, A. (2024) · The 2024 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3630106.3658956 · Cited by 144
“In the case of PaLM (the LLM that underwrites Google’s Bard), 50% of the data used to pre-train the model came from social media conversations [13], which could potentially over-represent populations with internet access [6].”
BackgroundChenais, G., Lagarde, E., & Gil-Jardiné, C. (2023) · Journal of Medical Internet Research · doi:10.2196/40031 · Cited by 142
“Several factors lead to better patient satisfaction in emergency medicine such as actual waiting times [83], perceived waiting times [84], staff-patient communication, and staff empathy and compassion [85].”
Mapping the ethics of generative AI: a comprehensive scoping review
Hagendorff, T. (2024) · Minds and Machines · doi:10.1007/s11023-024-09694-w · Cited by 141
“Moreover, there are concerns regarding the imposition of values embedded in AI systems on cultures distinct from those where the systems were developed (Bender et al., 2021; Wang et al., 2023b).”
InfluentialTowards trustworthy LLMs: a review on debiasing and dehallucinating in large language models
Lin, Z., Guan, S., Zhang, W., Zhang, H., Li, Y., & Zhang, H. (2024) · Artificial Intelligence Review · doi:10.1007/s10462-024-10896-y · Cited by 141
Gmyrek, P., Berg, J., Bescond, D., & International Labour Organization. Research Department (2023) · doi:10.54394/fhem8239 · Cited by 140
“Sceptics tend to believe that these machines are nothing more than “stochastic parrots” – powerful text summarizers, incapable of “learning” and producing original content, with little future for general purpose use and unsustainable computing costs (Bender et al. 2021).”
BackgroundChatClimate: Grounding conversational AI in climate science
Vaghefi, S. A., Stammbach, D., Muccione, V., Bingler, J., Ni, J., Kraus, M., Allen, S., Colesanti-Senni, C., Wekhof, T., Schimanski, T., et al. (2023) · Communications Earth & Environment · doi:10.1038/s43247-023-01084-x · Cited by 135
Kim, S. S. Y., Liao, Q. V., Vorvoreanu, M., Ballard, S., & Vaughan, J. W. (2024) · The 2024 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3630106.3658941 · Cited by 147
Carobene, A., Padoan, A., Cabitza, F., Banfi, G., & Plebani, M. (2024) · Clinical Chemistry and Laboratory Medicine (CCLM) · doi:10.1515/cclm-2023-1136 · Cited by 137
Bridging the Gulf of Envisioning: Cognitive Challenges in Prompt Based Interactions with LLMs
Subramonyam, H., Pea, R., Pondoc, C., Agrawala, M., & Seifert, C. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642754 · Cited by 162
“Another predominant technique used in prompting is chain-of-thought, or providing a series of reasoning steps to show the model how to get to the fnal answer [84, 178].”
BackgroundMethodAI Empire: Unraveling the interlocking systems of oppression in generative AI's global order
Tacheva, J., & Ramasubramanian, S. (2023) · Big Data & Society · doi:10.1177/20539517231219241 · Cited by 138
“Instead, it requires examining AI as a product of historical, geopolitical, economic, environmental, cultural, racial, gender, and class factors (Bender et al., 2021; Benjamin, 2019; Crawford, 2021; Noble, 2018).”
BackgroundGunasekar, S., Zhang, Y., Aneja, J., Mendes, C. C. T., Del Giorno, A., Gopi, S., Javaheripi, M., Kauffmann, P., de Rosa, G., Saarikivi, O., et al. (2023) · arXiv · doi:10.48550/arxiv.2306.11644 · Cited by 138
Eight Things to Know about Large Language Models
Bowman, S. R. (2023) · arXiv · doi:10.48550/arxiv.2304.00612 · Cited by 137
“…arise with the development of deployment of large-scale AI systems, including issues around environmental impacts, access, mis-use, privacy, safety, and the concentration of power (Amodei et al., 2016; Bender et al., 2021; Bommasani et al., 2021; Birhane et al., 2022; Weidinger et al., 2022, i.a.).”
The User Experience of ChatGPT: Findings from a Questionnaire Study of Early Users
Skjuve, M., Følstad, A., & Brandtzaeg, P. B. (2023) · Proceedings of the 5th International Conference on Conversational User Interfaces · doi:10.1145/3571884.3597144 · Cited by 135
Assessing the utility of ChatGPT throughout the entire clinical workflow
Rao, A., Pang, M., Kim, J., Kamineni, M., Lie, W., Prasad, A. K., Landman, A., Dreyer, K. J., & Succi, M. D. (2023) · doi:10.1101/2023.02.21.23285886 · Cited by 134
Could a Large Language Model be Conscious?
Chalmers, D. J. (2023) · arXiv · doi:10.48550/arxiv.2303.07103 · Cited by 143
Large language models and agricultural extension services
Tzachor, A., Devare, M., Richards, C. E., Pypers, P., Ghosh, A., Koo, J., Johal, S., & King, B. (2023) · Nature Food · doi:10.1038/s43016-023-00867-x · Cited by 200
Brinkmann, L., Baumann, F., Bonnefon, J.-F., Derex, M., Müller, T. F., Nussberger, A.-M., Czaplicka, A., Acerbi, A., Griffiths, T. L., Henrich, J., et al. (2023) · Nature Human Behaviour · doi:10.1038/s41562-023-01742-2 · Cited by 132
“Irrespective of whether LLMs truly exhibit understanding 71 , the mere recombination of cultural traits can lead to innovation and fuel cultural evolution 69,72 .”
BackgroundMethodInfluentialEthical and regulatory challenges of Generative AI in education: a systematic review
García-López, I. M., & Trujillo-Liñán, L. (2025) · Frontiers in Education · doi:10.3389/feduc.2025.1565938 · Cited by 144
Black Box Warning: Large Language Models and the Future of Infectious Diseases Consultation
Schwartz, I. S., Link, K. E., Daneshjou, R., & Cortés-Penfield, N. (2024) · Clinical Infectious Diseases · doi:10.1093/cid/ciad633 · Cited by 131
Lee, V. R., Pope, D., Miles, S., & Zárate, R. C. (2024) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2024.100253 · Cited by 130
Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learned
Ganguli, D., Lovitt, L., Kernion, J., Askell, A., Bai, Y., Kadavath, S., Mann, B., Perez, E., Schiefer, N., Ndousse, K., et al. (2022) · arXiv · doi:10.48550/arxiv.2209.07858 · Cited by 130
“…social biases (e.g., [47, 28, 1, 33, 7]), generating offensive or toxic outputs [25], leaking personally identifiable information from the training data [13], aiding in disinformation campaigns [12], generating extremist texts [37], spreading falsehoods [35], and more [9, 10, 18, 57, 22, 51].”
BackgroundBias and Unfairness in Information Retrieval Systems: New Challenges in the LLM Era
Dai, S., Chen, X., Xu, S., Pang, L., Dong, Z., & Xu, J. (2024) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3637528.3671458 · Cited by 331
“Specifically, some low-quality, factual errors, and long-distance repetition in the training texts harm the factual correctness of the text generated by LLMs [10, 74, 88].”
BackgroundIbrahim, N., AboulEla, S., Ibrahim, A. A., & Kashef, R. F. (2024) · Discover Artificial Intelligence · doi:10.1007/s44163-024-00175-8 · Cited by 209
Storey, V. C., Yue, W. T., Zhao, J. L., & Lukyanenko, R. (2025) · Information Systems Frontiers · doi:10.1007/s10796-025-10581-7 · Cited by 296
“Large language models contain billions of parameters making it impossible for humans to fully understand how they make their decisions (Bender et al., 2021).”
InfluentialGPT-3-Driven Pedagogical Agents to Train Children’s Curious Question-Asking Skills
Abdelghani, R., Wang, Y.-H., Yuan, X., Wang, T., Lucas, P., Sauzéon, H., & Oudeyer, P.-Y. (2024) · International Journal of Artificial Intelligence in Education · doi:10.1007/s40593-023-00340-7 · Cited by 129
A Theory for Emergence of Complex Skills in Language Models
Arora, S., & Goyal, A. (2023) · arXiv · doi:10.48550/arxiv.2307.15936 · Cited by 129
“Teach a man to read and write, and you have put into his hands the great keys of the wisdom box.”
BackgroundHigher Education's Generative Artificial Intelligence Paradox: The Meaning of Chatbot Mania
Rudolph, J., Mohamed Ismail, F. M., & Popenici, S. (2024) · Journal of University Teaching and Learning Practice · doi:10.53761/54fs5e77 · Cited by 189
“The bots are but “stochastic parrots” (Bender et al., 2021).”
Reducing the Carbon Impact of Generative AI Inference (today and in 2035)
Chien, A. A., Lin, L., Nguyen, H., Rao, V., Sharma, T., & Wijayawardana, R. (2023) · Proceedings of the 2nd Workshop on Sustainable Computer Systems · doi:10.1145/3604930.3605705 · Cited by 128
Ethical ChatGPT: Concerns, Challenges, and Commandments
Zhou, J., Müller, H., Holzinger, A., & Chen, F. (2024) · Electronics · doi:10.3390/electronics13173417 · Cited by 128
Sezgin, E., Sirrianni, J., & Linwood, S. L. (2022) · JMIR Medical Informatics · doi:10.2196/32875 · Cited by 127
“Given these biases, it would be unwise to deploy GPT-3 or any other sizable language model without active bias testing [15].”
BackgroundMethodIn Conversation with Artificial Intelligence: Aligning language Models with Human Values
Kasirzadeh, A., & Gabriel, I. (2023) · Philosophy & Technology · doi:10.1007/s13347-023-00606-x · Cited by 127
Social Dynamics of AI Support in Creative Writing
Gero, K. I., Long, T., & Chilton, L. B. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3580782 · Cited by 129
The Rise of Artificial Intelligence in Educational Measurement: Opportunities and Ethical Challenges
Bulut, O., & Beiting-Parrish, M. (2024) · Chinese/English Journal of Educational Measurement and Evaluation · doi:10.59863/miql7785 · Cited by 126
“LLMs may also generate items containing unfamiliar or inappropriate terms for underrepresented groups or groups using low-resource languages, raising fairness concerns (Bender et al., 2021).”
Societal Biases in Language Generation: Progress and Challenges
Sheng, E., Chang, K.-W., Natarajan, P., & Peng, N. (2021) · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.18653/v1/2021.acl-long.330 · Cited by 131
“Furthermore, by filtering out all words deemed “bad”, Bender et al. (2021) warns that we remove the discourse of marginalized populations.”
BackgroundAcademic publisher guidelines on AI usage: A ChatGPT supported thematic analysis
Perkins, M., & Roe, J. (2024) · F1000Research · doi:10.12688/f1000research.142411.2 · Cited by 123
Kirk, H. R., Vidgen, B., Röttger, P., & Hale, S. A. (2024) · Nature Machine Intelligence · doi:10.1038/s42256-024-00820-y · Cited by 122
Generative AI and the politics of visibility
Gillespie, T. (2024) · Big Data & Society · doi:10.1177/20539517241252131 · Cited by 122
“…of reliable information; the human labor on which they depend; their hidden environmental costs; the copyright implications of their training data; their economic impact on creative employment; and the biases they sometimes demonstrate (Bender et al., 2021; Crawford, 2021; Weidinger et al., 2022).”
Generative artificial intelligence and engineering education
Johri, A., Katz, A. S., Qadir, J., & Hingle, A. (2023) · Journal of Engineering Education · doi:10.1002/jee.20537 · Cited by 121
“For instance, LLMs have been likened to “ stochastic parrots ” (Bender et al., 2021) since they work by “ haphazardly stitching together sequences of linguistic forms it has observed in its vast training data, according to probabilistic information about how they combine, but without any reference…”
BackgroundTypology of Risks of Generative Text-to-Image Models
Bird, C., Ungless, E., & Kasirzadeh, A. (2023) · Proceedings of the 2023 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3600211.3604722 · Cited by 121
Al-Abdullatif, A. M., & Alsubaie, M. A. (2024) · Behavioral Sciences · doi:10.3390/bs14090845 · Cited by 126
The limitations and ethical considerations of ChatGPT
Hua, S., Jin, S., & Jiang, S. (2024) · Data Intelligence · doi:10.1162/dint_a_00243 · Cited by 120
“Besides, added to its “random parrot” feature [22], ChatGPT can easily generate false and outdated information and thus mislead users.”
BackgroundHarms of Gender Exclusivity and Challenges in Non-Binary Representation in Language Technologies
Dev, S., Monajatipoor, M., Ovalle, A., Subramonian, A., Phillips, J., & Chang, K.-W. (2021) · Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2021.emnlp-main.150 · Cited by 123
“Text representations have been known to learn and exacerbate skewed associations and social biases from underlying data (Zhao et al., 2017; Bender et al., 2021; Dev, 2020), thus propagating representational harm.”
Widder, D. G., & Nafus, D. (2023) · Big Data & Society · doi:10.1177/20539517231177620 · Cited by 119
“Echoing criticism of endless AI scale (Bender et al., 2021), Gebru and Hanna propose such a model of AI development, where the goal is not to produce “ AI for the value of AI itself, ” but to instead be “ sensitive to other forms of knowledge ” in order to examine and curate data sets even if this…”
BackgroundTheory Is All You Need: AI, Human Cognition, and Causal Reasoning
(2024) · doi:10.1287/stsc.2024.0189 · Cited by 151
Ogueji, K., Zhu, Y., & Lin, J. (2021) · Workshop on Multilingual Representation Learning (MRL) · doi:10.18653/v1/2021.mrl-1.11 · Cited by 352
“Potential Ethical Benefits: Recent works have called for more considerations of ethics and related concerns in the development of pretrained language models (Bender et al., 2021).”
BackgroundThe Data-Production Dispositif
Miceli, M., & Posada, J. (2022) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3555561 · Cited by 118
“sponds to the voracious demand for more, cheaper, and increasingly differentiated data to feed the growing AI industry [5, 13].”
Background“Kelly is a Warm Person, Joseph is a Role Model”: Gender Biases in LLM-Generated Reference Letters
Wan, Y., Pu, G., Sun, J., Garimella, A., Chang, K.-W., & Peng, N. (2023) · Findings of the Association for Computational Linguistics: EMNLP 2023 · doi:10.18653/v1/2023.findings-emnlp.243 · Cited by 136
Large language models: a survey of their development, capabilities, and applications
Yadagiri, A., & Pakray, P. (2024) · Knowledge and Information Systems · doi:10.1007/s10115-024-02310-4 · Cited by 218
Sætra, H. S. (2022) · Sustainable Development · doi:10.1002/sd.2438 · Cited by 161
ChatGPT Undermines Human Reflexivity, Scientific Responsibility and Responsible Management Research
Lindebaum, D., & Fleming, P. (2024) · British Journal of Management · doi:10.1111/1467-8551.12781 · Cited by 116
Shah, C., & Bender, E. M. (2022) · ACM SIGIR Conference on Human Information Interaction and Retrieval · doi:10.1145/3498366.3505816 · Cited by 115
Self-citationEthics for the majority world: AI and the question of violence at scale
Ricaurte, P. (2022) · Media, Culture & Society · doi:10.1177/01634437221099612 · Cited by 116
“centres and training algorithms (Bender et al., 2021; Thompson et al., 2020), polluting territories and displacing communities (Ricaurte and Ciacci, 2020; Tapia and Peña, 2020).”
BackgroundExploiting Programmatic Behavior of LLMs: Dual-Use Through Standard Security Attacks
Kang, D., Li, X., Stoica, I., Guestrin, C., Zaharia, M., & Hashimoto, T. (2024) · 2024 IEEE Security and Privacy Workshops (SPW) · doi:10.1109/spw63631.2024.00018 · Cited by 114
“Our work furthers the study of harms and risks of LLMs (Bender et al., 2021; Bommasani et al., 2021; Liang et al., 2022; Abid et al., 2021; Gehman et al., 2020; Ganguli et al., 2022; Weidinger et al., 2021; 2022), focusing on demonstrating their potential of being actively misused.”
BackgroundExploiting Programmatic Behavior of LLMs: Dual-Use Through Standard Security Attacks
Kang, D., Li, X., Stoica, I., Guestrin, C., Zaharia, M., & Hashimoto, T. (2023) · arXiv · doi:10.48550/arxiv.2302.05733 · Cited by 114
“Our work furthers the study of harms and risks of LLMs (Bender et al., 2021; Bommasani et al., 2021; Liang et al., 2022; Abid et al., 2021; Gehman et al., 2020; Ganguli et al., 2022; Weidinger et al., 2021; 2022), focusing on demonstrating their potential of being actively misused.”
BackgroundA toolbox for surfacing health equity harms and biases in large language models
Pfohl, S. R., Cole-Lewis, H., Sayres, R., Neal, D., Asiedu, M., Dieng, A., Tomasev, N., Rashid, Q. M., Azizi, S., Rostamzadeh, N., et al. (2024) · Nature Medicine · doi:10.1038/s41591-024-03258-2 · Cited by 113
On the creativity of large language models
Franceschelli, G., & Musolesi, M. (2025) · AI & SOCIETY · doi:10.1007/s00146-024-02127-3 · Cited by 113
“LLMs have already been analyzed (and sometimes criticized) from different perspectives, e.g., fairness (Bender et al., 2021), concept understanding (Bender and Koller, 2020), societal impact (Tamkin et al., 2021), and anthropomorphism (Shanahan, 2024b) just to name a few.”
How large language models can reshape collective intelligence
Burton, J. W., Lopez-Lopez, E., Hechtlinger, S., Rahwan, Z., Aeschbach, S., Bakker, M. A., Becker, J. A., Berditchevskaia, A., Berger, J., Brinkmann, L. U. L., et al. (2024) · Nature Human Behaviour · doi:10.1038/s41562-024-01959-9 · Cited by 245
Rethinking search: making domain experts out of dilettantes
Metzler, D., Tay, Y., Bahri, D., & Najork, M. (2021) · ACM SIGIR Forum · doi:10.1145/3476415.3476428 · Cited by 112
“Pre-trained LMs are trained to maximize their predictive power on their training data, and thus they may reflect societal biases in that data [Bender et al., 2021; Hutchinson et al., 2020; Sheng et al., 2019].”
BackgroundMethodGuo, W., & Caliskan, A. (2021) · Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3461702.3462536 · Cited by 180
Peer review of GPT-4 technical report and systems card
Gallifant, J., Fiske, A., Levites Strekalova, Y. A., Osorio-Valencia, J. S., Parke, R., Mwavu, R., Martinez, N., Gichoya, J. W., Ghassemi, M., Demner-Fushman, D., et al. (2024) · PLOS Digital Health · doi:10.1371/journal.pdig.0000417 · Cited by 111
Fakour, H., & Imani, M. (2025) · Frontiers in Education · doi:10.3389/feduc.2025.1528603 · Cited by 112
A Watermark for Large Language Models
Kirchenbauer, J., Geiping, J., Wen, Y., Katz, J., Miers, I., & Goldstein, T. (2023) · arXiv · doi:10.48550/arxiv.2301.10226 · Cited by 194
“A watermark is a hidden pattern in text that is imperceptible to humans, while making the text algorithmically identifiable as synthetic.”
Challenges in Detoxifying Language Models
Welbl, J., Glaese, A., Uesato, J., Dathathri, S., Mellor, J., Hendricks, L. A., Anderson, K., Kohli, P., Coppin, B., & Huang, P.-S. (2021) · Findings of the Association for Computational Linguistics: EMNLP 2021 · doi:10.18653/v1/2021.findings-emnlp.210 · Cited by 112
Liesenfeld, A., Lopez, A., & Dingemanse, M. (2023) · Proceedings of the 5th International Conference on Conversational User Interfaces · doi:10.1145/3571884.3604316 · Cited by 111
“It is one of the achievements of publicly funded science that it can afford to not jump on the bandwagon and instead make room for reflection [2, 5].”
BackgroundThe Diagnostic and Triage Accuracy of the GPT-3 Artificial Intelligence Model
Levine, D. M., Tuwani, R., Kompa, B., Varma, A., Finlayson, S. G., Mehrotra, A., & Beam, A. (2023) · doi:10.1101/2023.01.30.23285067 · Cited by 106
Ong, J. C. L., Seng, B. J. J., Law, J. Z. F., Low, L. L., Kwa, A. L. H., Giacomini, K. M., & Ting, D. S. W. (2024) · Cell Reports Medicine · doi:10.1016/j.xcrm.2023.101356 · Cited by 111
“52,64 Particular to LMICs, the engagement of low-cost workers to perform annotation of toxic language (including text embedding hate speech, sexual abuse, violence) may contribute to negative psychological health outcomes among these workers.”
BackgroundDiversity and language technology: how language modeling bias causes epistemic injustice
Helm, P., Bella, G., Koch, G., & Giunchiglia, F. (2024) · Ethics and Information Technology · doi:10.1007/s10676-023-09742-6 · Cited by 110
“When it comes to questions of linguistic bias in NLP, Hovy and Prabhumoye (2021) identify five sources of bias: (1) the data,(2) the annotation process, (3) the models, (4) the input representation process and (5) the research design for studying bias.”
InfluentialA Recipe for Arbitrary Text Style Transfer with Large Language Models
Reif, E., Ippolito, D., Yuan, A., Coenen, A., Callison-Burch, C., & Wei, J. (2022) · Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) · doi:10.18653/v1/2022.acl-short.94 · Cited by 109
“Large LM safety concerns Large LMs themselves come with their own host of difficulties, barriers to entry, and potential safety concerns as discussed by Bender et al. (2021), which are also valid for this style transfer method.”
BackgroundLarge Language Models for Business Process Management: Opportunities and Challenges
Vidgof, M., Bachhofner, S., & Mendling, J. (2023) · Lecture notes in business information processing · doi:10.1007/978-3-031-41623-1_7 · Cited by 208
Assessment of chemistry knowledge in large language models that generate code
White, A. D., Hocky, G. M., Gandhi, H. A., Ansari, M., Cox, S., Wellawatte, G. P., Sasmal, S., Yang, Z., Liu, K., Singh, Y., et al. (2023) · Digital Discovery · doi:10.1039/d2dd00087c · Cited by 108
Guidelines for ethical use and acknowledgement of large language models in academic writing
Porsdam Mann, S., Vazirani, A. A., Aboy, M., Earp, B. D., Minssen, T., Cohen, I. G., & Savulescu, J. (2024) · Nature Machine Intelligence · doi:10.1038/s42256-024-00922-7 · Cited by 108
Prompting meaning: a hermeneutic approach to optimising prompt engineering with ChatGPT
Henrickson, L., & Meroño-Peñuela, A. (2023) · AI & Society · doi:10.1007/s00146-023-01752-8 · Cited by 179
“…about the specific aspects that these leaderboards incentivise, noting the leaderboards’ historical focuses on performance at the expense of other qualities that NLP and other research communities value (e.g. fairness, energy efficiency) (e.g. Bender et al. 2021; Ethayarajh and Jurafsky 2020).”
Ethics Sheet for Automatic Emotion Recognition and Sentiment Analysis
Mohammad, S. M. (2022) · Computational Linguistics · doi:10.1162/coli_a_00433 · Cited by 107
“See Bender et al. (2021) for ethical considerations in the use of large language models, including: • Labeled Training and Testing Data: AER systems often make use of a relatively small number of example instances that are manually labeled (annotated) for emotions.”
Background
Metadata
- Authors
- Bender, E. M., Gebru, T., McMillan-Major, A., & Shmitchell, S.
- Affiliations
- University of Washington (US)
- Black in AI, Palo Alto, CA, USA
- The Aether
- Venue
- Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency
- Publisher
- ACM
- Published
- 2021-03
- Pages
- 610-623
- OpenAlex
- W3133702157
- Cited by
- 6,988
- References
- 117
- Citations by year
- 2026: 2484 · 2025: 1887 · 2024: 1153 · 2023: 842 · 2022: 384 · 2021: 229
- FWCI
- 300.72
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- ACM Conference on Fairness, Accountability, and Transparency (FAccT) (publishedVersion, cc-by)
- Creighton Digital Repository (CDR) (submittedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Keywords
- large language models, model size, AI risks, financial costs, data curation, dataset documentation, stakeholder value, BERT, GPT, fine-tuning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.