On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜
Open accessThe DOI matches its Crossref record; 156 of the 159 references it cites were found in a registry or an index, and 3 are not checked yet.
Abstract
The past 3 years of work in NLP have been characterized by the development and deployment of ever larger language models, especially for English. BERT, its variants, GPT-2/3, and others, most recently Switch-C, have pushed the boundaries of the possible both through architectural innovations and through sheer size. Using these pretrained models and the methodology of fine-tuning them for specific tasks, researchers have extended the state of the art on a wide array of tasks as measured by leaderboards on specific benchmarks for English. In this paper, we take a step back and ask: How big is too big? What are the possible risks associated with this technology and what paths are available for mitigating those risks? We provide recommendations including weighing the environmental and financial costs first, investing resources into curating and carefully documenting datasets rather than ingesting everything on the web, carrying out pre-development exercises evaluating how the planned approach fits into research and development goals and supports stakeholder values, and encouraging research directions beyond ever larger language models.
Checked 2026-11-22 · 3 references not checked yet
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The text we found for this paper looks like another document, so the self-check wasn’t run.
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325 · cited 5×
“[39] in particular noted that training on a large dataset and fine-tuning for specific tasks leads to strictly increasing results on the GLUE tasks [138] for English as the hyperparameters of the model were increased.”
BackgroundMethodResultInfluentialGlove: Global Vectors for Word Representation
Pennington, J., Socher, R., & Manning, C. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1162 · Cited by 34,071
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372 · cited 4×
“documents linked to from Reddit [25], plus Wikipedia and a collection”
BackgroundMethodInfluentialRoBERTa: A Robustly Optimized BERT Pretraining Approach
Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., & Stoyanov, V. (2019) · arXiv · doi:10.48550/arxiv.1907.11692 · Cited by 16,515
Distilling the Knowledge in a Neural Network
Hinton, G., Vinyals, O., & Dean, J. (2015) · arXiv · doi:10.48550/arxiv.1503.02531 · Cited by 14,204 · cited 1×
“[26, 58], quantization [118, 153], factorized embedding parameterization and cross-layer parameter sharing [70], and progressive module replacing [146].”
Deep Contextualized Word Representations
Peters, M., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., & Zettlemoyer, L. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1202 · Cited by 11,983
Proceedings of the 24th international conference on Machine learning
(2007) · doi:10.1145/1273496 · Cited by 11,630
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., & Liu, P. J. (2019) · arXiv · doi:10.48550/arxiv.1910.10683 · Cited by 10,929
The Mathematical Theory of Communication
Shannon, C. E., Weaver, W., & Wiener, N. (1950) · Physics Today · doi:10.1063/1.3067010 · Cited by 9,801
Research works printed but not linked: 48
- [4] American Psychological Association. 2019. Discrimination: What it is, and how to cope. https://www.apa.org/topics/discrimination (2019). Google Scholar
- 10.18653/v1 · DOI
- 10.18653/v1 · DOI
- [15] Ruha Benjamin. 2019. Race After Technology: Abolitionist Tools for the New Jim Code. Polity Press, Cambridge, UK. Crossref Google Scholar
- [16] Elettra Bietti and Roxana Vatanparast. 2020. Data Waste. Harvard International Law Journal 61 (2020). Google Scholar
- 10.18653/v1 · DOI
- 10.18653/v1 · DOI
- 10.1016/0010-0277(86) · doi:10.1016/0010-0277(86
- [38] Terrance de Vries, Ishan Misra, Changhan Wang, and Laurens van der Maaten. 2019. Does object recognition work for everyone?. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. 52--59. Google Scholar
- 10.18653/v1 · DOI
- [42] Ethan Fast, Tina Vachovsky, and Michael Bernstein. 2016. Shirtless and Dangerous: Quantifying Linguistic Signals of Gender Bias in an Online Fiction Writing Community. In Proceedings of the International AAAI Conference on Web and Social Media, Vol. 10. Google Scholar
- 10.18653/v1 · DOI
- [45] Darja Fišer, Ruihong Huang, Vinodkumar Prabhakaran, Rob Voigt, Zeerak Waseem, and Jacqueline Wernimont (Eds.). 2018. Proceedings of the 2nd Workshop on Abusive Language Online (ALW2). Association for Computational Linguistics, Brussels, Belgium. https://www.aclweb.org/anthology/W18-5100 Google Scholar
- [50] Batya Friedman, Peter H. Kahn, Jr., and Alan Borning. 2006. Value sensitive design and information systems. In Human-Computer Interaction in Management Information Systems: Foundations, P Zhang and D Galletta (Eds.). M. E. Sharpe, Armonk NY, 348--372. Google Scholar
- 10.18653/v1 · DOI
- 10.18653/v1 · DOI
- 10.18653/v1 · DOI
- [67] Brendan Kennedy, Drew Kogon, Kris Coombs, Joseph Hoover, Christina Park, Gwenyth Portillo-Wightman, Aida Mostafazadeh Davani, Mohammad Atari, and Morteza Dehghani. 2018. A typology and coding manual for the study of hate-based rhetoric. PsyArXiv. July 18 (2018). Google Scholar
- [77] Marianna Martindale and Marine Carpuat. 2018. Fluency Over Adequacy: A Pilot Study in Measuring User Trust in Imperfect MT. In Proceedings of the 13th Conference of the Association for Machine Translation in the Americas (Volume 1: Research Track). Association for Machine Translation in the Americas, Boston, MA, 13--25. https://www.aclweb.org/anthology/W18-1803 Google Scholar
- [81] Douglas M McLeod. 2007. News coverage and social protest: How the media's protect paradigm exacerbates social conflict. Journal of Dispute Resolution (2007), 185. Google Scholar
- [87] Robert C. Moore and William Lewis. 2010. Intelligent Selection of Language Model Training Data. In Proceedings of the ACL 2010 Conference Short Papers. Association for Computational Linguistics, Uppsala, Sweden, 220--224. https://www.aclweb.org/anthology/P10-2041 Google Scholar
- 10.18653/v1 · DOI
- [92] Maggie Nelson. 2015. The Argonauts. Graywolf Press, Minneapolis. Google Scholar
- 10.18653/v1 · DOI
- [96] David Ortiz, Daniel Myers, Eugene Walls, and Maria-Elena Diaz. 2005. Where do we stand with newspaper data? Mobilization: An International Quarterly 10, 3 (2005), 397--419. Google Scholar
- 10.3115/v1 · doi:10.3115/v1
- 10.18653/v1 · DOI
- 10.18653/v1 · DOI
- [106] Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language models are unsupervised multitask learners. OpenAI Blog 1, 8 (2019), 9. Google Scholar
- 10.18653/v1 · DOI
- [109] Sarah T. Roberts, Joel Tetreault, Vinodkumar Prabhakaran, and Zeerak Waseem (Eds.). 2019. Proceedings of the Third Workshop on Abusive Language Online. Association for Computational Linguistics, Florence, Italy. https://www.aclweb.org/anthology/W19-3500 Google Scholar
- [112] Corby Rosset. 2020. Turing-NLG: A 17-billion-parameter language model by Microsoft. Microsoft Blog (2020). Google Scholar
- 10.18653/v1 · DOI
- 10.18653/v1 · DOI
- [121] Joonbo Shin, Yoonhyung Lee, and Kyomin Jung. 2019. Effective Sentence Scoring Method Using BERT for Speech Recognition. In Asian Conference on Machine Learning. 1081--1093. Google Scholar
- [124] Karen Spärck Jones. 2004. Language modelling's generative model: Is it rational? Technical Report. Computer Laboratory, University of Cambridge. Google Scholar
- [127] Katrina Srigley and Lorraine Sutherland. 2019. Decolonizing, Indigenizing, and Learning Biskaaybiiyang in the Field: Our Oral History Journey1. The Oral History Review (2019). Google Scholar
- 10.18653/v1 · DOI
- 10.18653/v1 · DOI
- 10.18653/v1 · DOI
- 10.18653/v1 · DOI
- 10.18653/v1 · DOI
- 10.18653/v1 · DOI
- 10.18653/v1 · DOI
- [150] Zhilin Yang, Zihang Dai, Yiming Yang, Jaime Carbonell, Russ R Salakhutdinov, and Quoc V Le. 2019. Xlnet: Generalized autoregressive pretraining for language understanding. In Advances in neural information processing systems. 5753--5763. Google Scholar
- 10.18653/v1 · DOI
- 10.18653/v1 · DOI
- 10.18653/v1 · DOI
Cited by
Items 801–900 of 6,931
Generative AI and science communication in the physical sciences
Biyela, S., Dihal, K., Gero, K. I., Ippolito, D., Menczer, F., Schäfer, M. S., & Yokoyama, H. (2024) · Nature Reviews Physics · doi:10.1038/s42254-024-00691-7 · Cited by 40
Africa, ChatGPT, and Generative AI Systems: Ethical Benefits, Concerns, and the Need for Governance
Wakunuma, K., & Eke, D. (2024) · Philosophies · doi:10.3390/philosophies9030080 · Cited by 33
Start Generating: Harnessing Generative Artificial Intelligence for Sociological Research
Davidson, T. (2024) · Socius Sociological Research for a Dynamic World · doi:10.1177/23780231241259651 · Cited by 76
“Returning to the questions of positionality discussed above, such a model takes a position that recycles hegemonic tropes present in its training data (Bender et al. 2021).”
InfluentialPashangpour, S., & Nejat, G. (2024) · Robotics · doi:10.3390/robotics13080112 · Cited by 50
“However, it is important that healthcare providers understand that the LLM-augmented healthcare robots cannot understand the data which they have embedded and are merely generating/identifying patterns which were present during training [173].”
BackgroundNext Generation XR Systems—Large Language Models Meet Augmented and Virtual Reality
Afzal, M. Z., Ali, S. A., Stricker, D., Eisert, P., Hilsmann, A., Pérez-Marcos, D., Bianchi, M., Crottaz‐Herbette, S., Ioris, R. D., Mangina, E., et al. (2025) · IEEE Computer Graphics and Applications · doi:10.1109/mcg.2025.3548554 · Cited by 41
Companionship in code: AI's role in the future of human connection
Andersson, M. (2025) · Humanities and Social Sciences Communications · doi:10.1057/s41599-025-05536-x · Cited by 39
“…this behaviour stems from probabilistic weighting based on conversational history and stylistic alignment, and although the label ‘ stochastic parrots ’ (Bender et al., 2021) applies to both ELIZA and advanced LLMs for their lack of true language understanding, the above response illustrates just…”
Unpacking the Expressed Consequences of AI Research in Broader Impact Statements
Nanayakkara, P., Hullman, J., & Diakopoulos, N. (2021) · Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3461702.3462608 · Cited by 32
“Suggestions like this one are timely, as the computer science research community—as well as the broader public—is becoming increasingly aware of the ways in which deployed technologies have disproportionate negative impacts on marginalized communities [7, 17, 32] and pose significant costs to the environment [6, 39, 4…”
Malekzadeh, M., Borovykh, A., & Gündüz, D. (2021) · ACM SIGSAC Conference on Computer and Communications Security (CCS) · doi:10.1145/3460120.3484533 · Cited by 53
“The costs and potential risks associated with large-scale language models, such as discriminatory biases, are discussed in [5] and the research community is encouraged to consider the impacts of the ever-increasing size of DNNs beyond just the model’s accuracy for a target task.”
BackgroundYour fairness may vary: Pretrained language model fairness in toxic text classification
Baldini, I., Wei, D., Ramamurthy, K. N., Singh, M., & Yurochkin, M. (2022) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2022.findings-acl.176 · Cited by 80
“Bender et al. (2021) claim that this same property may also lead to more pronounced biases in their predictions, as the large data that LMs are trained on is not curated.”
BackgroundContextualizing the ethics of algorithms: A socio-professional approach
Avnoon, N., Kotliar, D. M., & Bahir, S. R. (2023) · New Media & Society · doi:10.1177/14614448221145728 · Cited by 46
Lalor, J. P., Abbasi, A., Oketch, K., Yang, Y., & Forsgren, N. (2024) · ACM Transactions on Information Systems · doi:10.1145/3641276 · Cited by 58
“In the context of PLMs, models trained on large historical corpora using techniques such as word2vec [70], GloVe [77], and contextual embeddings such as BERT [37] have been found to capture historical stereotypes related to gender and race [15, 31, 33, 44].”
BackgroundMethodControversies, contradiction, and “participation” in AI
Sloane, M. (2024) · Big Data & Society · doi:10.1177/20539517241235862 · Cited by 48
“A recent, prominent case of this phenomenon includes the controversy induced by the fi ring of Google researcher Timnit Gebru, a Black woman, for her work on AI ethics, speci fi cally debunking natural language processing systems (Bender et al., 2021; Metz and Wakabayashi, 2020).”
Generative AI has a language problem
Choudhury, M. (2023) · Nature Human Behaviour · doi:10.1038/s41562-023-01716-4 · Cited by 33
“The world is currently grappling with debates over the capabilities of LLMs, whether they exhibit ‘sparks of artificial general intelligence’ 1 and the kind of harm they can perpetrate 2 .”
BackgroundFuture Shock: Generative AI and the International AI Policy and Governance Crisis
Leslie, D., & Perini, A. M. (2024) · Harvard Data Science Review · doi:10.1162/99608f92.88b4cc98 · Cited by 33
Mobilising new frontiers in digital transformation research: A problematization review
Ashrafi, A., Constantinides, P., Mehandjiev, N., & Thatcher, J. B. (2025) · Information Systems Journal · doi:10.1111/isj.12531 · Cited by 33
“Thus, such data homogenization may not represent diverse communities, perspectives, and beliefs (Bender et al., 2021).”
Hickman, L., Dunlop, P. D., & Wolf, J. L. (2024) · International Journal of Selection and Assessment · doi:10.1111/ijsa.12479 · Cited by 33
Navigating the Risks of Using Large Language Models for Text Annotation in Social Science Research
Lin, H., & Zhang, Y. (2026) · Social Science Computer Review · doi:10.1177/08944393251366243 · Cited by 33
“It is important to note that we are not AI-Luddites; as advancements continue, we anticipate improvements that may mitigate these limitations and challenges, enabling more effective and ethical use of LLMs in social science research.”
Hickman, L., Dunlop, P. D., & Wolf, J. L. (2024) · doi:10.31234/osf.io/9cs23 · Cited by 33
Kenthapadi, K., Sameki, M., & Taly, A. (2024) · Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining · doi:10.1145/3637528.3671467 · Cited by 36
Benjamin, J. J., Lindley, J., Edwards, E., Rubegni, E., Korjakow, T., Grist, D., & Sharkey, R. (2024) · Designing Interactive Systems Conference · doi:10.1145/3643834.3660677 · Cited by 32
Artificial Intelligence’s new clothes? A system technology perspective
Vannuccini, S., & Prytkova, E. (2023) · Journal of Information Technology · doi:10.1177/02683962231197824 · Cited by 51
“In turn, Language Models (LM) (regardless of their architectural complexity) are systems that perform string prediction – that is, statistical prediction of a character, word, or phrase – given preceding and/or surrounding text (Bender et al., 2021).”
BackgroundBig Data, Proxies, Algorithmic Decision‐Making and the Future of Management Theory
Lindebaum, D., Möser, C., & Islam, G. (2023) · Journal of Management Studies · doi:10.1111/joms.13032 · Cited by 43
Reconstructing AI Ethics Principles: Rawlsian Ethics of Artificial Intelligence
Westerstrand, S. (2024) · Science and Engineering Ethics · doi:10.1007/s11948-024-00507-y · Cited by 48
“…least advantaged (e.g., discriminatory systems used to make life-changing decisions in Tilmes, 2022; AI systems that undermine minorities in, e.g., Bender et al., 2021; Janssen et al., 2022; or AI systems eroding societal structures that protect fundamental rights in Coeckelbergh, 2024), which…”
BackgroundFactors influencing trust in algorithmic decision-making: an indirect scenario-based experiment
Marmolejo‐Ramos, F., Marrone, R. L., Korolkiewicz, M. W., Gabriel, F. C., Siemens, G., Joksimovíc, S., Yamada, Y., Mori, Y., Rahwan, T., Sahakyan, M., et al. (2025) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2024.1465605 · Cited by 52
Survey on Factuality in Large Language Models
Wang, C., Liu, X., Yue, Y., Guo, Q., Hu, X., Tang, X., Zhang, T., Cheng, J., Yao, Y., Hu, X., et al. (2025) · ACM Computing Surveys · doi:10.1145/3742420 · Cited by 399
“Yet, with their unparalleled capabilities, concerns have arisen about the potential of LLMs to generate non-factual or misleading content [7, 10, 135].”
Large Language Models are not Models of Natural Language: they are Corpus Models.
Veres, C. (2021) · IEEE Access · doi:10.1109/access.2022.3182505 · Cited by 27
“While the most powerful systems have shown remarkable abilities complex language tasks like question answering and writing prose about arbitrary topics [7], there are contrary claims that ANNs are nothing but giant stochastic parrots, with many hidden dangers if misrepresented as systems that understand language in an…”
BackgroundFairness in Language Models Beyond English: Gaps and Challenges
Ramesh, K., Sitaram, S., & Choudhury, M. (2023) · Findings of the Association for Computational Linguistics: EACL 2023 · doi:10.18653/v1/2023.findings-eacl.157 · Cited by 36
Testing of Detection Tools for AI-Generated Text
Weber-Wulff, D., Anohina-Naumeca, A., Bjelobaba, S., Foltýnek, T., Guerrero-Dib, J., Popoola, O., Šigut, P., & Waddington, L. (2023) · arXiv · doi:10.48550/arxiv.2306.15666 · Cited by 32
Warr, M., & Heath, M. K. (2025) · Journal of Teacher Education · doi:10.1177/00224871251325073 · Cited by 35
“Bender et al. (2021) cautioned against the hege-monic view implicit in large language models (LLMs), and Benjamin (2019) argued that the perceived objectivity of ML algorithms obscures encoded oppression.”
Can ChatGPT Help in Electronics Research and Development? A Case Study with Applied Sensors
Tafferner, Z., Balázs, B., Krammer, O., & Géczy, A. (2023) · Sensors · doi:10.3390/s23104879 · Cited by 31
Lewicki, K., Lee, M. S. A., Cobbe, J., & Singh, J. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581463 · Cited by 30
Heinz, M. V., Bhattacharya, S., Trudeau, B. M., Quist, R. E., Song, S. H., Lee, C. M., & Jacobson, N. C. (2023) · Digital Health · doi:10.1177/20552076231170499 · Cited by 41
“Some researchers argue that large language models may even be dangerous, producing human-like, comprehensible, fl uent language, without basis in common sense or rational judgment —“ stochastic parrots, ” 8 or “ a mouth without a brain.”
BackgroundA new role for translators and trainers: MT literacy consultants
Ehrensberger‐Dow, M., Benites, A. D., & Lehr, C. (2023) · The Interpreter and Translator Trainer · doi:10.1080/1750399x.2023.2237328 · Cited by 73
Can Large Language Models Assist in Hazard Analysis?
Diemert, S., & Weber, J. H. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-40953-0_35 · Cited by 67
Kroon, A. C., Welbers, K., Trilling, D., & van Atteveldt, W. (2023) · Communication Methods and Measures · doi:10.1080/19312458.2023.2261372 · Cited by 44
“The environmental impact of training models that are ever-increasing in size is substantial and warrants careful consideration (Bender et al., 2021).”
BackgroundMethodInfluentialApplying Generative Artificial Intelligence to cognitive models of decision making
Malloy, T., & González, C. (2024) · Frontiers in Psychology · doi:10.3389/fpsyg.2024.1387948 · Cited by 54
“This is due in part to the potential of biases present in language processing and generating models such as Large Language Models (LLMs) (Bender et al., 2021).”
Scott, I. A., & Zuccon, G. (2024) · Internal Medicine Journal · doi:10.1111/imj.16393 · Cited by 39
Flitcroft, M. A., Sheriff, S. A., Wolfrath, N., Maddula, R., McConnell, L., Xing, Y., Haines, K., Wong, S. L., & Kothari, A. N. (2024) · Annals of Surgical Oncology · doi:10.1245/s10434-024-15549-6 · Cited by 42
“15,16 Finally, the continuous advancement of AI necessitates a reassessment of traditional citation practices and the recognition dynamics in academia, commonly known as the "Matthew Effect."”
The sociolinguistic foundations of language modeling
Grieve, J., Bartl, S., Fuoli, M., Grafmiller, J., Huang, W., Jawerbaum, A., Murakami, A., Perlman, M., Rœmling, D., & Winter, B. (2025) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2024.1472411 · Cited by 89
“Stereotyping and related forms of discrimination generated by LLMs have also often been assumed to result from careless data collection and a lack of data curation [10].”
InfluentialExploring the scope of generative AI in literature review development
Schryen, G., Marrone, M., & Yang, J. (2025) · Electronic Markets · doi:10.1007/s12525-025-00754-2 · Cited by 75
“As a result, they have been described as “stochastic parrots” that build sentences from data traces (Bender et al., 2021).”
Two types of AI existential risk: decisive and accumulative
Kasirzadeh, A. (2025) · Philosophical Studies · doi:10.1007/s11098-025-02301-3 · Cited by 115
Harvey, E., Koenecke, A., & Kizilcec, R. F. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713210 · Cited by 86
“These capabilities bring significant risks as well, as outlined by Bender et al. [5] and Weidinger et al. [80] in two widely cited taxonomies.”
Influential'I'm Categorizing LLM as a Productivity Tool': Examining Ethics of LLM Use in HCI Research Practices
Kapania, S., Wang, R., Li, T. J.-J., Li, T., & Shen, H. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3711000 · Cited by 81
“Over the last few years, progress in NLP has been characterized by the development of larger language models [4] requiring substantial computational resources and access to extensive datasets that have been challenging to obtain outside big companies and well-funded research labs.”
Eight Things to Know about Large Language Models
Bowman, S. R. (2024) · Critical AI · doi:10.1215/2834703x-11556011 · Cited by 122
“…arise with the development of deployment of large-scale AI systems, including issues around environmental impacts, access, mis-use, privacy, safety, and the concentration of power (Amodei et al., 2016; Bender et al., 2021; Bommasani et al., 2021; Birhane et al., 2022; Weidinger et al., 2022, i.a.).”
Nicmanis, M., & Spurrier, H. (2025) · International Journal of Qualitative Methods · doi:10.1177/16094069251354863 · Cited by 46
“…based on probabilities from their training data, inherently prioritising central tendencies (the mean) and overlooking less frequent or unexpected insights (the tail-end), which is a substantial limitation for the depth required in Big-Q research (Bender et al., 2021; Vaswani et al., 2017).”
Conformism, Ignorance & Injustice: AI as a Tool of Epistemic Oppression
Miragoli, M. (2025) · Episteme · doi:10.1017/epi.2024.11 · Cited by 31
“Similarly, recent surveys of Wikipedians find that only 8.8 – 15% are women or girls ’ (Bender et al .”
InfluentialComparing Sentence-Level Suggestions to Message-Level Suggestions in AI-Mediated Communication
Fu, L., Newman, B., Jakesch, M., & Kreps, S. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581351 · Cited by 31
AI Writing Assistants Influence Topic Choice in Self-Presentation
Poddar, R., Sinha, R., Naaman, M., & Jakesch, M. (2023) · Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544549.3585893 · Cited by 30
Fraile Navarro, D., Kocaballi, A. B., Dras, M., & Berkovsky, S. (2023) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3569893 · Cited by 28
Porcaro, L., Gómez, E., & Castillo, C. (2022) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3512956 · Cited by 35
“Indeed, AI systems — an umbrella term wherein also Recommender Systems can be included [40] — have already been proven to reinforce hegemonic biases and discrimination in different applications, in fields such as Computer Vision [60] and Natural Language Processing [5].”
BackgroundAddressing the Unsustainability of Deep Neural Networks with Next-Gen AI
Vallentin, A. G., Þórisson, K. R., & Latapie, H. M. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-33469-6_30 · Cited by 30
Legal Means of Providing the Principle of Transparency of the Artificial Intelligence
Kharitonova, Y. S. (2023) · Journal of Digital Technologies and Law · doi:10.21202/jdtl.2023.14 · Cited by 46
Disambiguating Algorithmic Bias: From Neutrality to Justice
Edenberg, E., & Wood, A. (2023) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1145/3600211.3604695 · Cited by 46
Mapping Texts: Computational Text Analysis for the Social Sciences
Stoltz, D. S., & Taylor, M. A. (2023) · doi:10.1093/oso/9780197756874.001.0001 · Cited by 41
Fontana De Vargas, M., Yu, C., Shane, H. C., & Moffatt, K. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642080 · Cited by 30
Academics’ Weak(ening) Resistance to Generative AI: The Cause and Cost of Prestige?
Watermeyer, R., Lanclos, D., Phipps, L., Shapiro, H., Guizzo, D., & Knight, C. (2024) · Postdigital Science and Education · doi:10.1007/s42438-024-00524-x · Cited by 59
“They are also, however, discredited for their ‘sto-chastic’ disposition (Bender et al. 2021) while they are blamed for compromising an academic rewards system, exacerbating research output saturation and lowering research quality, placing further strain on ethical review and peer-review processes…”
The advantages of lexicon-based sentiment analysis in an age of machine learning
van der Veen, A. M., & Bleich, E. (2025) · PLoS ONE · doi:10.1371/journal.pone.0313092 · Cited by 63
Davidson, T., & Karell, D. (2025) · Sociological Methods & Research · doi:10.1177/00491241251339184 · Cited by 58
Scaccia, J., & Scott, V. C. (2021) · Implementation Science · doi:10.1186/s13012-021-01120-4 · Cited by 37
“Analytic approaches involving deep semantic modeling with word vectorization are a step forward though are not without their own limitations [39].”
BackgroundVashishth, S., Newman-Griffis, D. R., Joshi, R., Dutt, R., & Rosé, C. P. (2021) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2021.103880 · Cited by 62
“Potential effects of biased data on MEDTYPE and novel datasets The effects of biased data and algorithms in producing biased AI systems (including medical AI systems) is an important and rapidly-growing area of inquiry [96,97].”
BackgroundRyoo, J. J., Morris, A., & Margolis, J. (2021) · ACM Transactions on Computing Education · doi:10.1145/3453653 · Cited by 32
“In December 2020, Dr. Timnit Gebru was fired from Google when she wanted to publicly share her research revealing the racist and sexist biases of the company’s Artificial Intelligence (AI) systems [2].”
BackgroundLouie, R., Engel, J., & Huang, C.-Z. A. (2022) · 27th International Conference on Intelligent User Interfaces · doi:10.1145/3490099.3511159 · Cited by 30
Markedness in Visual Semantic AI
Wolfe, R., & Caliskan, A. (2022) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3531146.3533183 · Cited by 71
Debiasing Pre-Trained Language Models via Efficient Fine-Tuning
Gira, M., Zhang, R.-S., & Lee, K. (2022) · LTEDI · doi:10.18653/v1/2022.ltedi-1.8 · Cited by 50
Multimodal large language models for inclusive collaboration learning tasks
Lewis, A. (2022) · Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies: Student Research Workshop · doi:10.18653/v1/2022.naacl-srw.26 · Cited by 30
“, 2021), while others pose ethical, environmental, and interpretability concerns about unbounded scaling of model size (Bender et al., 2021; Strubell et al., 2020; Weidinger et al., 2021).”
BackgroundA psychometric framework for evaluating and shaping personality traits in large language models
Serapio-García, G., Safdari, M., Crepy, C., Sun, L., Fitz, S., Romero, P., Abdulhai, M., Faust, A., & Matarić, M. (2025) · Nature Machine Intelligence · doi:10.1038/s42256-025-01115-6 · Cited by 30
Epistemic fragmentation poses a threat to the governance of online targeting
Milano, S., Mittelstadt, B., Wachter, S., & Russell, C. (2021) · Nature Machine Intelligence · doi:10.1038/s42256-021-00358-3 · Cited by 29
Nee, J., Smith, G. M., Sheares, A. M., & Rustagi, I. (2022) · Big Data & Society · doi:10.1177/20539517221090930 · Cited by 50
“On Twitter, pervasive harassment of women, especially Black women, may lead to self-censorship ( Toxic, n.d. ), reducing the representation of (Black) women ’ s view-points in NLP datasets (Bender et al., 2021).”
BackgroundMethodInfluentialSabbata, S. D., Ballatore, A., Miller, H. J., Sieber, R. E., Tyukin, I., & Yeboah, G. (2023) · International Journal of Geographical Information Systems · doi:10.1080/13658816.2023.2279978 · Cited by 51
“…of the ethical concerns raised by foundation models is beyond the scope of this editorial, we refer the reader to the ‘stochastic parrots’ paper by Bender et al. (2021), which has become foundational for a critical understanding of LLMs, and the useful taxonomy of both upstream and downstream…”
Birkenmaier, L., Lechner, C. M., & Wagner, C. (2023) · Communication Methods and Measures · doi:10.1080/19312458.2023.2285765 · Cited by 55
“Thus, the promises of LLMs carry the danger that communication researchers trust the output of these models blindly, without accounting for the various types of biases and limitations inherent to LLMs (as discussed by Bender et al., 2021).”
Deconstructing the Digital Infrastructures Supporting Archaeological Knowledge
Huggett, J. (2024) · Current Swedish Archaeology · doi:10.37718/csa.2023.01 · Cited by 33
“…and consumers, thinking infrastructures remove this clear separation (Bowker et al. 2019:9), and by extension, knowledge producers may become the systems themselves, as large language models applied in natural language processing increasingly risk being perceived (for example Bender et al. 2021).”
BackgroundAmirova, A., Fteropoulli, T., Ahmed, N., Cowie, M., & Leibo, J. (2023) · PLoS ONE · doi:10.1371/journal.pone.0300024 · Cited by 36
“Care must be taken in the conditioning of LLMs and the interpretation of LLM-generated outputs to avoid perpetuating harmful biases (Bender et al., 2021).”
BackgroundTowards interculturally adaptive conversational AI
Brandt, A., & Hazel, S. (2024) · Applied Linguistics Review · doi:10.1515/applirev-2024-0187 · Cited by 48
“2 Current challenges of language and culture in conversational AI It is widely accepted that many aspects of AI technologies replicate the prejudices which exist in society, for example racial and gender biases, whether this is in relation to machine learning (Mehrabi et al. 2021), large language models (LLMs, such as…”
BackgroundLarge models of what? Mistaking engineering achievements for human linguistic agency
Birhane, A., & McGann, M. (2024) · Language Sciences · doi:10.1016/j.langsci.2024.101672 · Cited by 99
Examining the responsible use of zero-shot AI approaches to scoring essays
Johnson, M., & Zhang, M. (2024) · Scientific Reports · doi:10.1038/s41598-024-79208-2 · Cited by 55
Cui, Y., & Hai, Z. (2025) · Education and Information Technologies · doi:10.1007/s10639-025-13448-1 · Cited by 48
“The theoretical component should help students understand the strengths, limitations, and potential risks of AIGC, such as AI illusions, where AI systems generate plausible yet false information (Bender et al., 2021).”
Generative AI lacks the human creativity to achieve scientific discovery from scratch
Ding, A. W., & Li, S. (2025) · Scientific Reports · doi:10.1038/s41598-025-93794-9 · Cited by 63
The Role of AI-Based Chatbots in Public Health Emergencies: A Narrative Review
Branda, F., Stella, M., Ceccarelli, C., Cabitza, F., Ceccarelli, G., Maruotti, A., Ciccozzi, M., & Scarpa, F. (2025) · Future Internet · doi:10.3390/fi17040145 · Cited by 51
“ChatGPT, while very adept at generating human-like text, can sometimes produce “hallucinations”, generating plausible but factually incorrect information [89].”
From Prompt Engineering to Prompt Science with Humans in the Loop
Shah, C. S. (2025) · Communications of the ACM · doi:10.1145/3709599 · Cited by 49
The cultural stereotype and cultural bias of ChatGPT
Yuan, H., Che, Z., Zhang, Y., Shao, L. H., Yuan, X., Huang, L., Hu, X., Peng, K., & Luo, S. (2025) · Journal of Pacific Rim Psychology · doi:10.1177/18344909251355673 · Cited by 37
“Different LLMs vary in training data, architecture, and fi ne-tuning processes, potentially producing distinct cultural bias patterns (Blodgett et al., 2020; Bender et al., 2021).”
ELLMA-T: an Embodied LLM-agent for Supporting English Language Learning in Social VR
Pan, M., Kitson, A., Wan, H., & Prpa, M. (2025) · Designing Interactive Systems Conference · doi:10.1145/3715336.3735786 · Cited by 76
Where Do Stories Come From? Examining the Exploration Process in Investigative Data Journalism
Showkat, D., & Baumer, E. P. S. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3479534 · Cited by 29
Reliable and Safe Use of Machine Translation in Medical Settings
Mehandru, N., Robertson, S., & Salehi, N. (2022) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3531146.3533244 · Cited by 76
Poetry Will Not Optimize; or, What Is Literature to AI?
Elam, M. (2023) · American Literature · doi:10.1215/00029831-10575077 · Cited by 56
The Humanities in the Digital: Beyond Critical Digital Humanities
Viola, L. (2023) · Springer eBooks · doi:10.1007/978-3-031-16950-2 · Cited by 50
Jiang, Y., Zhang, L., & Wang, W. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.220 · Cited by 69
“Since our method relies on pre-trained language models, it may run the danger of inheriting and propagating some of the models’ negative biases from the data they have been pre-trained on (Bender et al., 2021).”
MethodThe E.U.’s artificial intelligence act: an ordoliberal assessment
Wörsdörfer, M. (2023) · AI and Ethics · doi:10.1007/s43681-023-00337-x · Cited by 47
“1 As these discussions show, there appears to be a growing need for (some form of) AI regulation, and several governments have already begun taking initial measures.”
Coding Inequity: Assessing GPT-4’s Potential for Perpetuating Racial and Gender Biases in Healthcare
Zack, T., Lehman, E., Suzgun, M., Rodriguez, J. A., Celi, L. A., Gichoya, J., Jurafsky, D., Szolovits, P., Bates, D. W., Abdulnour, R.-E. E., et al. (2023) · medRxiv · doi:10.1101/2023.07.13.23292577 · Cited by 28
Llama 2: Early Adopters' Utilization of Meta's New Open-Source Pretrained ModelPreprint
Roumeliotis, K. I., Tselikas, N. D., & Nasiopoulos, D. Κ. (2023) · Preprints.org · doi:10.20944/preprints202307.2142.v2 · Cited by 51
Kirwan, A. (2024) · Irish Educational Studies · doi:10.1080/03323315.2023.2284901 · Cited by 29
“Bender et al. (2021) have warned of the dangers of larger and larger LLMs which include ‘ stereotyping, denigration, increases in extremist ideology ’ and other such issues as ‘ humans encounter seemingly coherent LM [sic.] output ’ (Bender et al. 2021, 619).”
Arakawa, R., Yakura, H., & Goto, M. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581133 · Cited by 29
Looking for a handsome carpenter! Debiasing GPT-3 job advertisements
Borchers, C., Gala, D., Gilburt, B., Oravkin, E., Bounsi, W., Asano, Y. M., & Kirk, H. (2022) · Proceedings of the 4th Workshop on Gender Bias in Natural Language Processing (GeBNLP) · doi:10.18653/v1/2022.gebnlp-1.22 · Cited by 29
Moral stereotyping in large language models
Zewail, A., Figueroa, A., Graham, J., & Atari, M. (2026) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2519941123 · Cited by 29
“As a result, LLMs are likely reflective of a dataset where Western, English texts are overweighted, and thus overrepresented in subsequent outputs (12).”
Inverse Scaling: When Bigger Isn't Better
McKenzie, I. R., Lyzhov, A., Pieler, M., Parrish, A., Mueller, A., Prabhu, A., McLean, E., Kirtland, A., Ross, A., Liu, A., et al. (2023) · arXiv · doi:10.48550/arxiv.2306.09479 · Cited by 29
“It is well-known that LMs replicate human biases such as racism and sexism (Bender et al., 2021).”
BackgroundIs Resistance Futile?: Early Career Game Developers, Generative AI, and Ethical Skepticism
Boucher, J. D., Smith, G., & Telliel, Y. D. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3641889 · Cited by 30
“The commodification of the artworks produced by both their peers and those in positions they aspire to via taking their work without consent for training data further exacerbates this divide [8].”
BackgroundHassoulas, A., Powell, N., Roberts, L., Umla-Runge, K., Gray, L., & Coffey, M. J. (2023) · Journal of Applied Learning and Teaching · doi:10.37074/jalt.2023.6.2.13 · Cited by 28
Ethical Reasoning over Moral Alignment: A Case and Framework for In-Context Ethical Policies in LLMs
Rao, A., Khandelwal, A., Tanmay, K., Agarwal, U., & Choudhury, M. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.892 · Cited by 136
Chinese Cyberbullying Detection Using XLNet and Deep Bi-LSTM Hybrid Model
Chen, S., Wang, J., & He, K. (2024) · Information · doi:10.3390/info15020093 · Cited by 39
“As suggested by Bender et al. [45], it is also important to consider the environmental and financial costs for the application of LLMs, as high costs can prevent some people from benefiting from the advances made by LLMs.”
RAG-Ex: A Generic Framework for Explaining Retrieval Augmented Generation
Sudhi, V., Bhat, S. R., Rudat, M., & Teucher, R. (2024) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3626772.3657660 · Cited by 63
Jones, R. H. (2024) · Applied Linguistics Review · doi:10.1515/applirev-2024-0188 · Cited by 44
Metadata
- Authors
- Bender, E. M., Gebru, T., McMillan-Major, A., & Shmitchell, S.
- Affiliations
- University of Washington (US)
- Black in AI, Palo Alto, CA, USA
- The Aether
- Venue
- Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency
- Publisher
- ACM
- Published
- 2021-03
- Pages
- 610-623
- OpenAlex
- W3133702157
- Cited by
- 6,988
- References
- 117
- Citations by year
- 2026: 2484 · 2025: 1887 · 2024: 1153 · 2023: 842 · 2022: 384 · 2021: 229
- FWCI
- 300.72
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- ACM Conference on Fairness, Accountability, and Transparency (FAccT) (publishedVersion, cc-by)
- Creighton Digital Repository (CDR) (submittedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Keywords
- large language models, model size, AI risks, financial costs, data curation, dataset documentation, stakeholder value, BERT, GPT, fine-tuning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.