On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜
Open accessThe DOI matches its Crossref record; 156 of the 159 references it cites were found in a registry or an index, and 3 are not checked yet.
Abstract
The past 3 years of work in NLP have been characterized by the development and deployment of ever larger language models, especially for English. BERT, its variants, GPT-2/3, and others, most recently Switch-C, have pushed the boundaries of the possible both through architectural innovations and through sheer size. Using these pretrained models and the methodology of fine-tuning them for specific tasks, researchers have extended the state of the art on a wide array of tasks as measured by leaderboards on specific benchmarks for English. In this paper, we take a step back and ask: How big is too big? What are the possible risks associated with this technology and what paths are available for mitigating those risks? We provide recommendations including weighing the environmental and financial costs first, investing resources into curating and carefully documenting datasets rather than ingesting everything on the web, carrying out pre-development exercises evaluating how the planned approach fits into research and development goals and supports stakeholder values, and encouraging research directions beyond ever larger language models.
Checked 2026-11-22 · 3 references not checked yet
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The text we found for this paper looks like another document, so the self-check wasn’t run.
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325 · cited 5×
“[39] in particular noted that training on a large dataset and fine-tuning for specific tasks leads to strictly increasing results on the GLUE tasks [138] for English as the hyperparameters of the model were increased.”
BackgroundMethodResultInfluentialGlove: Global Vectors for Word Representation
Pennington, J., Socher, R., & Manning, C. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1162 · Cited by 34,071
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372 · cited 4×
“documents linked to from Reddit [25], plus Wikipedia and a collection”
BackgroundMethodInfluentialRoBERTa: A Robustly Optimized BERT Pretraining Approach
Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., & Stoyanov, V. (2019) · arXiv · doi:10.48550/arxiv.1907.11692 · Cited by 16,515
Distilling the Knowledge in a Neural Network
Hinton, G., Vinyals, O., & Dean, J. (2015) · arXiv · doi:10.48550/arxiv.1503.02531 · Cited by 14,204 · cited 1×
“[26, 58], quantization [118, 153], factorized embedding parameterization and cross-layer parameter sharing [70], and progressive module replacing [146].”
Deep Contextualized Word Representations
Peters, M., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., & Zettlemoyer, L. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1202 · Cited by 11,983
Proceedings of the 24th international conference on Machine learning
(2007) · doi:10.1145/1273496 · Cited by 11,630
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., & Liu, P. J. (2019) · arXiv · doi:10.48550/arxiv.1910.10683 · Cited by 10,929
The Mathematical Theory of Communication
Shannon, C. E., Weaver, W., & Wiener, N. (1950) · Physics Today · doi:10.1063/1.3067010 · Cited by 9,801
Research works printed but not linked: 48
- [4] American Psychological Association. 2019. Discrimination: What it is, and how to cope. https://www.apa.org/topics/discrimination (2019). Google Scholar
- 10.18653/v1 · DOI
- 10.18653/v1 · DOI
- [15] Ruha Benjamin. 2019. Race After Technology: Abolitionist Tools for the New Jim Code. Polity Press, Cambridge, UK. Crossref Google Scholar
- [16] Elettra Bietti and Roxana Vatanparast. 2020. Data Waste. Harvard International Law Journal 61 (2020). Google Scholar
- 10.18653/v1 · DOI
- 10.18653/v1 · DOI
- 10.1016/0010-0277(86) · doi:10.1016/0010-0277(86
- [38] Terrance de Vries, Ishan Misra, Changhan Wang, and Laurens van der Maaten. 2019. Does object recognition work for everyone?. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. 52--59. Google Scholar
- 10.18653/v1 · DOI
- [42] Ethan Fast, Tina Vachovsky, and Michael Bernstein. 2016. Shirtless and Dangerous: Quantifying Linguistic Signals of Gender Bias in an Online Fiction Writing Community. In Proceedings of the International AAAI Conference on Web and Social Media, Vol. 10. Google Scholar
- 10.18653/v1 · DOI
- [45] Darja Fišer, Ruihong Huang, Vinodkumar Prabhakaran, Rob Voigt, Zeerak Waseem, and Jacqueline Wernimont (Eds.). 2018. Proceedings of the 2nd Workshop on Abusive Language Online (ALW2). Association for Computational Linguistics, Brussels, Belgium. https://www.aclweb.org/anthology/W18-5100 Google Scholar
- [50] Batya Friedman, Peter H. Kahn, Jr., and Alan Borning. 2006. Value sensitive design and information systems. In Human-Computer Interaction in Management Information Systems: Foundations, P Zhang and D Galletta (Eds.). M. E. Sharpe, Armonk NY, 348--372. Google Scholar
- 10.18653/v1 · DOI
- 10.18653/v1 · DOI
- 10.18653/v1 · DOI
- [67] Brendan Kennedy, Drew Kogon, Kris Coombs, Joseph Hoover, Christina Park, Gwenyth Portillo-Wightman, Aida Mostafazadeh Davani, Mohammad Atari, and Morteza Dehghani. 2018. A typology and coding manual for the study of hate-based rhetoric. PsyArXiv. July 18 (2018). Google Scholar
- [77] Marianna Martindale and Marine Carpuat. 2018. Fluency Over Adequacy: A Pilot Study in Measuring User Trust in Imperfect MT. In Proceedings of the 13th Conference of the Association for Machine Translation in the Americas (Volume 1: Research Track). Association for Machine Translation in the Americas, Boston, MA, 13--25. https://www.aclweb.org/anthology/W18-1803 Google Scholar
- [81] Douglas M McLeod. 2007. News coverage and social protest: How the media's protect paradigm exacerbates social conflict. Journal of Dispute Resolution (2007), 185. Google Scholar
- [87] Robert C. Moore and William Lewis. 2010. Intelligent Selection of Language Model Training Data. In Proceedings of the ACL 2010 Conference Short Papers. Association for Computational Linguistics, Uppsala, Sweden, 220--224. https://www.aclweb.org/anthology/P10-2041 Google Scholar
- 10.18653/v1 · DOI
- [92] Maggie Nelson. 2015. The Argonauts. Graywolf Press, Minneapolis. Google Scholar
- 10.18653/v1 · DOI
- [96] David Ortiz, Daniel Myers, Eugene Walls, and Maria-Elena Diaz. 2005. Where do we stand with newspaper data? Mobilization: An International Quarterly 10, 3 (2005), 397--419. Google Scholar
- 10.3115/v1 · doi:10.3115/v1
- 10.18653/v1 · DOI
- 10.18653/v1 · DOI
- [106] Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language models are unsupervised multitask learners. OpenAI Blog 1, 8 (2019), 9. Google Scholar
- 10.18653/v1 · DOI
- [109] Sarah T. Roberts, Joel Tetreault, Vinodkumar Prabhakaran, and Zeerak Waseem (Eds.). 2019. Proceedings of the Third Workshop on Abusive Language Online. Association for Computational Linguistics, Florence, Italy. https://www.aclweb.org/anthology/W19-3500 Google Scholar
- [112] Corby Rosset. 2020. Turing-NLG: A 17-billion-parameter language model by Microsoft. Microsoft Blog (2020). Google Scholar
- 10.18653/v1 · DOI
- 10.18653/v1 · DOI
- [121] Joonbo Shin, Yoonhyung Lee, and Kyomin Jung. 2019. Effective Sentence Scoring Method Using BERT for Speech Recognition. In Asian Conference on Machine Learning. 1081--1093. Google Scholar
- [124] Karen Spärck Jones. 2004. Language modelling's generative model: Is it rational? Technical Report. Computer Laboratory, University of Cambridge. Google Scholar
- [127] Katrina Srigley and Lorraine Sutherland. 2019. Decolonizing, Indigenizing, and Learning Biskaaybiiyang in the Field: Our Oral History Journey1. The Oral History Review (2019). Google Scholar
- 10.18653/v1 · DOI
- 10.18653/v1 · DOI
- 10.18653/v1 · DOI
- 10.18653/v1 · DOI
- 10.18653/v1 · DOI
- 10.18653/v1 · DOI
- 10.18653/v1 · DOI
- [150] Zhilin Yang, Zihang Dai, Yiming Yang, Jaime Carbonell, Russ R Salakhutdinov, and Quoc V Le. 2019. Xlnet: Generalized autoregressive pretraining for language understanding. In Advances in neural information processing systems. 5753--5763. Google Scholar
- 10.18653/v1 · DOI
- 10.18653/v1 · DOI
- 10.18653/v1 · DOI
Cited by
Items 101–200 of 6,931
Should ChatGPT be biased? Challenges and risks of bias in large language models
Ferrara, E. (2023) · First Monday · doi:10.5210/fm.v28i11.13346 · Cited by 282
“If the data used to train a language model contains biases, either from the source material or through the selection process, these biases can be absorbed by the model and subsequently reflected in its behavior [26, 27, 28, 29].”
BackgroundMethodInfluentialShould ChatGPT be Biased? Challenges and Risks of Bias in Large Language Models
Ferrara, E. (2023) · arXiv · doi:10.48550/arxiv.2304.03738 · Cited by 282
Risks and Benefits of Large Language Models for the Environment
Rillig, M. C., Ågerstrand, M., Bi, M., Gould, K. A., & Sauerland, U. (2023) · Environmental Science & Technology · doi:10.1021/acs.est.3c01106 · Cited by 270
AI Art and its Impact on Artists
Jiang, H. H., Brown, L., Cheng, J., Khan, M., Gupta, A., Workman, D., Hanna, A., Flowers, J., & Gebru, T. (2023) · Proceedings of the 2023 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3600211.3604681 · Cited by 265
“We echo [18]’s recommendations to use methodologies like value”
BackgroundMethodInfluentialSelf-citationDeveloping evaluative judgement for a time of generative artificial intelligence
Bearman, M., Tai, J., Dawson, P., Boud, D., & Ajjawi, R. (2024) · Assessment & Evaluation in Higher Education · doi:10.1080/02602938.2024.2335321 · Cited by 312
“The statistical tendency of generative AI towards dominant forms of knowledge and ways of seeing the world perpetuates existing inequities (Bender et al. 2021); the ‘silenced’ will have even less of a voice.”
Stadler, M., Bannert, M., & Sailer, M. (2024) · Computers in Human Behavior · doi:10.1016/j.chb.2024.108386 · Cited by 262
Casal, J. E., & Kessler, M. (2023) · Research Methods in Applied Linguistics · doi:10.1016/j.rmal.2023.100068 · Cited by 261
The Participatory Turn in AI Design: Theoretical Foundations and the Current State of Practice
Delgado, F., Yang, S., Madaio, M., & Yang, Q. (2023) · Equity and Access in Algorithms, Mechanisms, and Optimization · doi:10.1145/3617694.3623261 · Cited by 256
A Survey on Text Classification Algorithms: From Text to Predictions
Gasparetto, A., Marcuzzo, M., Zangari, A., & Albarelli, A. (2022) · Information · doi:10.3390/info13020083 · Cited by 253
Machine-learning-based evidence and attribution mapping of 100,000 climate impact studies
Callaghan, M., Schleussner, C.-F., Nath, S., Lejeune, Q., Knutson, T. R., Reichstein, M., Hansen, G., Theokritoff, E., Andrijevic, M., Brecha, R. J., et al. (2021) · Nature Climate Change · doi:10.1038/s41558-021-01168-6 · Cited by 251
The Fallacy of AI Functionality
Raji, I. D., Kumar, I. E., Horowitz, A., & Selbst, A. (2022) · 2022 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3531146.3533158 · Cited by 251
BERT applications in natural language processing: a review
Gardazi, N. M., Daud, A., Malik, M. K., Bukhari, A., Alsahfi, T., & Alshemaimri, B. (2025) · Artificial Intelligence Review · doi:10.1007/s10462-025-11162-5 · Cited by 251
“Such biases can influence downstream tasks in unintended ways (Bender et al. 2021).”
Co-Writing with Opinionated Language Models Affects Users’ Views
Jakesch, M., Bhat, A., Buschek, D., Zalmanson, L., & Naaman, M. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581196 · Cited by 265
Large language models for human–robot interaction: A review
Zhang, C., Chen, J., Li, J., Peng, Y., & Mao, Z. (2023) · Biomimetic Intelligence and Robotics · doi:10.1016/j.birob.2023.100131 · Cited by 254
Future directions for chatbot research: An interdisciplinary research agenda
Følstad, A., Araujo, T., Law, E. L.-C., Brandtzaeg, P. B., Papadopoulos, S., Reis, L., Baez, M., Laban, G., McAllister, P., Ischen, C., et al. (2021) · Computing · doi:10.1007/s00607-021-01016-7 · Cited by 247
AI Deception: A Survey of Examples, Risks, and Potential Solutions
Park, P. S., Goldstein, S., O'Gara, A., Chen, M., & Hendrycks, D. (2023) · arXiv · doi:10.48550/arxiv.2308.14752 · Cited by 243
Lau, S., & Guo, P. (2023) · Proceedings of the 2023 ACM Conference on International Computing Education Research V.1 · doi:10.1145/3568813.3600138 · Cited by 241
Human-level play in the game of Diplomacy by combining language models with strategic reasoning
Meta Fundamental AI Research Diplomacy Team (FAIR)†, Bakhtin, A., Brown, N., Dinan, E., Farina, G., Flaherty, C., Fried, D., Goff, A., Gray, J., Hu, H., et al. (2022) · Science · doi:10.1126/science.ade9097 · Cited by 241
The AI Mirror: How to Reclaim Our Humanity in an Age of Machine Thinking
Vallor, S. (2024) · doi:10.1093/oso/9780197759066.001.0001 · Cited by 239
The Edge of Sentience: Risk and Precaution in Humans, Other Animals, and AI
Birch, J. (2024) · doi:10.1093/9780191966729.001.0001 · Cited by 236
Mirowski, P., Mathewson, K. W., Pittman, J., & Evans, R. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581225 · Cited by 240
A Benchmark for Data Imputation Methods
Jäger, S., Allhorn, A., & Bießmann, F. (2021) · Frontiers in Big Data · doi:10.3389/fdata.2021.693674 · Cited by 231
“Moreover, poor data quality can foster unfair automated decisions, which marginalize minorities or have other negative societal impacts (Stoyanovich et al., 2020; Yang et al., 2020; Bender et al., 2021).”
BackgroundNortheastern University, Kostka, I., Toncelli, R., & Northeastern University (2023) · Teaching English as a Second or Foreign Language--TESL-EJ · doi:10.55593/ej.27107int · Cited by 235
“source of ChatGPT-generated text cannot be understood and biases from the source corpus dataset can be passed forward (Bender et al., 2021; Lo, 2023), students have an increased need for skills such as sorting through and evaluating information.”
BackgroundInfluentialThe Computational Limits of Deep Learning
Thompson, N., Greenewald, K., Lee, K., & Manso, G. F. (2023) · Ninth Computing within Limits 2023 · doi:10.21428/bf6fb269.1f033948 · Cited by 495
AI generates covertly racist decisions about people based on their dialect
Hofmann, V., Kalluri, P. R., Jurafsky, D., & King, S. (2024) · Nature · doi:10.1038/s41586-024-07856-5 · Cited by 233
Cain, W. (2024) · TechTrends · doi:10.1007/s11528-023-00896-0 · Cited by 232
“The “language model” part of the name refers to their purpose: to model, or understand, human language in a way that allows them to generate coherent and contextually appropriate responses (Bender et al., 2021).”
InfluentialNeural machine translation in foreign language teaching and learning: a systematic review
Klimova, B., Pikhart, M., Benites, A. D., Lehr, C., & Sanchez-Stockhammer, C. (2023) · Education and Information Technologies · doi:10.1007/s10639-022-11194-2 · Cited by 232
“However, such large language models are often prone to bias (Bender et al., 2021), e.g. regarding gender (cf. below), and they are mainly available for widespread, economically relevant languages (with English as the priority language due to its status as a global lingua franca), leading to much…”
BackgroundShort, C. E., & Short, J. C. (2023) · Journal of Business Venturing Insights · doi:10.1016/j.jbvi.2023.e00388 · Cited by 231
The uncontroversial ‘thingness’ of AI
Suchman, L. (2023) · Big Data & Society · doi:10.1177/20539517231206794 · Cited by 229
“In the fi eld of NLP, Bender et al. (2021: 611) distinguish between language understanding and ‘ string prediction tasks ’ over massive training datasets.”
BackgroundWhat is Human-Centered about Human-Centered AI? A Map of the Research Landscape
Capel, T., & Brereton, M. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3580959 · Cited by 228
Self-Diagnosis and Self-Debiasing: A Proposal for Reducing Corpus-Based Bias in NLP
Schick, T., Udupa, S., & Schütze, H. (2021) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00434 · Cited by 227
“Building training datasets with more care and deliberation, an alternative solution discussed by Bender et al. (2021), is important, especially for improving linguistic and cultural diversity in online and other forms of communication.”
BackgroundCLIPort: What and Where Pathways for Robotic Manipulation
Shridhar, M., Manuelli, L., & Fox, D. (2021) · arXiv · doi:10.48550/arxiv.2109.12098 · Cited by 223
AI deception: A survey of examples, risks, and potential solutions
Park, P. S., Goldstein, S., O’Gara, A., Chen, M., & Hendrycks, D. (2024) · Patterns · doi:10.1016/j.patter.2024.100988 · Cited by 243
Using ChatGPT in academic writing is (not) a form of plagiarism: What does the literature say?
Jarrah, A. M., Wardat, Y., & Fidalgo, P. (2023) · Online Journal of Communication and Media Technologies · doi:10.30935/ojcmt/13572 · Cited by 220
GeDi: Generative Discriminator Guided Sequence Generation
Krause, B., Gotmare, A. D., McCann, B., Keskar, N. S., Joty, S., Socher, R., & Rajani, N. F. (2021) · Findings of the Association for Computational Linguistics: EMNLP 2021 · doi:10.18653/v1/2021.findings-emnlp.424 · Cited by 220
“However, simply imitating the distribution of the training data during generation has many drawbacks (Bender et al., 2021); large-scale text training sets are crawled from the web, which is imbued with toxicity, bias, and misinformation.”
BackgroundTransformers in the Real World: A Survey on NLP Applications
Patwardhan, N., Marrone, S., & Sansone, C. (2023) · Information · doi:10.3390/info14040242 · Cited by 217
Huang, L., Yu, W., Ma, W., Zhong, W., Feng, Z., Wang, H., Chen, Q., Peng, W., Feng, X., Qin, B., et al. (2023) · arXiv · doi:10.48550/arxiv.2311.05232 · Cited by 2,516
“While scaling up pre-training data substantially enhances the competencies of LLMs (Kaplan et al., 2020; Hoffmann et al., 2022), challenges arise in maintaining consistent data quality, which can potentially introduce misinformation and biases (Bender et al., 2021; Weidinger et al., 2021).”
Delving into LLM-assisted writing in biomedical publications through excess vocabulary
Kobak, D., González-Márquez, R., Horvát, E.-Á., & Lause, J. (2024) · arXiv · doi:10.48550/arxiv.2406.07016 · Cited by 214
Machine Learning-Guided Protein Engineering
Kouba, P., Kohout, P., Haddadi, F., Bushuiev, A., Samusevich, R., Sedlar, J., Damborsky, J., Pluskal, T., Sivic, J., & Mazurenko, S. (2023) · ACS Catalysis · doi:10.1021/acscatal.3c02743 · Cited by 209
Post-hoc Interpretability for Neural NLP: A Survey
Madsen, A., Reddy, S., & Chandar, S. (2023) · ACM Computing Surveys · doi:10.1145/3546577 · Cited by 207
“For many of these applications, neural models have been shown to exhibit unwanted biases and similar ethical issues [16, 20, 42, 75, 82, 100].”
BackgroundSynthetic Replacements for Human Survey Data? The Perils of Large Language Models
Bisbee, J., Clinton, J. D., Dorff, C., Kenkel, B., & Larson, J. M. (2024) · Political Analysis · doi:10.1017/pan.2024.5 · Cited by 209
“…the characterization of public opinion and the creation of synthetic data, connecting with a growing body of research that asks similar questions (Bender et al. 2021; Spirling 2023) and documents similar issues (Abdulhai et al. 2023; Cao et al. 2023; Motoki, Pinho Neto, and Rodrigues 2023;…”
Bias in Large Language Models: Origin, Evaluation, and Mitigation
Guo, Y., Guo, M., Su, J., Yang, Z., Zhu, M., Li, H., Qiu, M., & Liu, S. S. (2024) · arXiv · doi:10.48550/arxiv.2411.10915 · Cited by 208
Evaluation of Text Generation: A Survey
Celikyilmaz, A., Clark, E., & Gao, J. (2020) · arXiv · doi:10.48550/arxiv.2006.14799 · Cited by 207
How to cheat on your final paper: Assigning AI for student writing
Fyfe, P. (2023) · AI & SOCIETY · doi:10.1007/s00146-022-01397-z · Cited by 205
“As scholars have recently suggested of such language models, “large datasets based on texts from the Internet overrepresent hegemonic viewpoints and encode biases potentially damaging to marginalized populations” (Bender et al. 2021, 1).”
BackgroundPredictability and Surprise in Large Generative Models
Ganguli, D., Hernandez, D., Lovitt, L., Askell, A., Bai, Y., Chen, A., Conerly, T., Dassarma, N., Drain, D., Elhage, N., et al. (2022) · 2022 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3531146.3533229 · Cited by 207
“However, similarly to [72], we do not consider here the costs of human labor involved in creating and annotating training data [28], the ethics of supply chains involved in creating the requisite hardware on which to train models [18], or the environmental costs of training models [7, 50, 62, 66].”
BackgroundSafe Latent Diffusion: Mitigating Inappropriate Degeneration in Diffusion Models
Schramowski, P., Brack, M., Deiseroth, B., & Kersting, K. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.02157 · Cited by 219
“These concerns include, for instance, models producing stereotypical and derogatory content [3] and gender and racial biases [10, 24, 38, 41].”
BackgroundTaleBrush: Sketching Stories with Generative Pretrained Language Models
Chung, J. J. Y., Kim, W., Yoo, K. M., Lee, H., Adar, E., & Chang, M. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3501819 · Cited by 206
“To alleviate this problem, we can try pretraining big language models with balanced data [9] or having a controllable module that tries to control these biases [76, 91].”
MethodResultOh, N., Choi, G.-S., & Lee, W. Y. (2023) · Annals of Surgical Treatment and Research · doi:10.4174/astr.2023.104.5.269 · Cited by 201
“4% accuracy rate on the Korean surgical board exam, it is important to remember that LLMs are generative models, often referred to as “stochastic parrots” [11].”
Levine, D. M., Tuwani, R., Kompa, B., Varma, A., Finlayson, S. G., Mehrotra, A., & Beam, A. (2024) · The Lancet Digital Health · doi:10.1016/s2589-7500(24)00097-9 · Cited by 199
Putting ChatGPT’s Medical Advice to the (Turing) Test: Survey Study
Nov, O., Singh, N., & Mann, D. (2023) · JMIR Medical Education · doi:10.2196/46939 · Cited by 198
“Research to mitigate risks of AI chatbot–generated responses, including the potential for patient harm caused by incorrect answers; cybersecurity vulnerabilities [31]; and environmental, social, and financial risks [32] should also be further explored.”
BackgroundMedical large language models are vulnerable to data-poisoning attacks
Alber, D. A., Yang, Z., Alyakin, A., Yang, E., Rai, S., Valliani, A. A., Zhang, J., Rosenbaum, G. R., Amend-Thomas, A. K., Kurland, D. B., et al. (2025) · Nature Medicine · doi:10.1038/s41591-024-03445-1 · Cited by 242
Challenges for Higher Education in the Era of Widespread Access to Generative AI
Walczak, K., & Cellary, W. (2023) · Economics and Business Review · doi:10.18559/ebr.2023.2.743 · Cited by 196
“Additionally the high cost of training LLMs may exac - erbate the digital divide by concentrating AI capabilities within well-funded organizations and institutions (Bender et al., 2021).”
BackgroundInfluentialA Framework for Evaluating and Disclosing the ESG Related Impacts of AI with the SDGs
Sætra, H. S. (2021) · Sustainability · doi:10.3390/su13158503 · Cited by 193
“For example, issues related to the carbon footprint of using AI and Big Data to train large natural language models are now problematized [8,49,58].”
BackgroundThe great Transformer: Examining the role of large language models in the political economy of AI
Luitse, D., & Denkena, W. (2021) · Big Data & Society · doi:10.1177/20539517211047734 · Cited by 190
“Second, training and running LLMs requires signi fi cant amounts of processing power (Strubell et al., 2019; Bender et al., 2021).”
BackgroundInfluentialBias of AI-generated content: an examination of news produced by large language models
Fang, X., Che, S., Mao, M., Zhang, H., Zhao, M., & Zhao, X. (2024) · Scientific Reports · doi:10.1038/s41598-024-55686-2 · Cited by 194
Foundations & Trends in Multimodal Machine Learning: Principles, Challenges, and Open Questions
Liang, P. P., Zadeh, A., & Morency, L.-P. (2024) · ACM Computing Surveys · doi:10.1145/3656580 · Cited by 195
“Co-regularization methods employ a regularizer that penalizes functions from either modality that disagree with each other.”
MethodKar, A. K., Varsha, P. S., & Rajan, S. (2023) · Global Journal of Flexible Systems Management · doi:10.1007/s40171-023-00356-x · Cited by 192
“Several ethical concerns are identified by ChatGPT-3 and are specifically addressed by OpenAI and divergent AI ethicists (Bender et al., 2021; Chiu & Alexander, 2021; Lucy & Bamman, 2021) focusing on the two aspects of manipulation and bias (Chan, 2023).”
BackgroundMethodInternet-Augmented Dialogue Generation
Komeili, M., Shuster, K., & Weston, J. (2022) · Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2022.acl-long.579 · Cited by 192
Delving into LLM-assisted writing in biomedical publications through excess vocabulary
Kobak, D., González-Márquez, R., Horvát, E.-Á., & Lause, J. (2025) · Science Advances · doi:10.1126/sciadv.adt3813 · Cited by 213
“Furthermore, LLMs can mimic biases and other deficiencies from their training data (Bender et al., 2021; Nav-igli et al., 2023; Bai et al., 2024; Choudhury, 2023), or even outright plagiarise (McCoy et al., 2023).”
Pataranutaporn, P., Liu, R., Finn, E., & Maes, P. (2023) · Nature Machine Intelligence · doi:10.1038/s42256-023-00720-7 · Cited by 192
“We argue that the observable factors of the AI agent comprise only half of the story; the force of imagination is also at play, allowing humans to construct a mental model of the world 49–56 .”
BackgroundFriha, O., Amine Ferrag, M., Kantarci, B., Cakmak, B., Ozgun, A., & Ghoualmi-Zine, N. (2024) · IEEE Open Journal of the Communications Society · doi:10.1109/ojcoms.2024.3456549 · Cited by 189
Beyond Goldfish Memory: Long-Term Open-Domain Conversation
Xu, J., Szlam, A., & Weston, J. (2022) · Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2022.acl-long.356 · Cited by 188
Parameter-efficient fine-tuning in large language models: a survey of methodologies
Wang, L., Chen, S., Jiang, L., Pan, S., Cai, R., Yang, S., & Yang, F. (2025) · Artificial Intelligence Review · doi:10.1007/s10462-025-11236-4 · Cited by 186
“However, they can also produce hallucinations, outputs that seem plausible but contain factual inaccuracies or misleading information [46], [47].”
On the Limits of Artificial Intelligence (AI) in Education
Selwyn, N. (2024) · Nordisk tidsskrift for pedagogikk og kritikk · doi:10.23865/ntpk.v10.6062 · Cited by 185
Automated Fact-Checking for Assisting Human Fact-Checkers
Nakov, P., Corney, D., Hasanain, M., Alam, F., Elsayed, T., Barrón-Cedeño, A., Papotti, P., Shaar, S., & Da San Martino, G. (2021) · Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2021/619 · Cited by 183
“The dangers of bias in large language models is becoming increasingly obvious [Bender et al., 2021], and should not be ignored just because the purpose of the system is benevolent.”
BackgroundPerspectives on Large Language Models for Relevance Judgment
Faggioli, G., Dietz, L., Clarke, C. L. A., Demartini, G., Hagen, M., Hauff, C., Kando, N., Kanoulas, E., Potthast, M., Stein, B., et al. (2023) · Proceedings of the 2023 ACM SIGIR International Conference on Theory of Information Retrieval · doi:10.1145/3578337.3605136 · Cited by 184
A generalist medical language model for disease diagnosis assistance
Liu, X., Liu, H., Yang, G., Jiang, Z., Cui, S., Zhang, Z., Wang, H., Tao, L., Sun, Y., Song, Z., et al. (2025) · Nature Medicine · doi:10.1038/s41591-024-03416-6 · Cited by 183
“Furthermore, the output of generative language models may contain factual errors, logic inconsistencies and problems with coherence 28 .”
ChatGPT’s inconsistent moral advice influences users’ judgment
Krügel, S., Ostermaier, A., & Uhl, M. (2023) · Scientific Reports · doi:10.1038/s41598-023-31341-0 · Cited by 183
Tree of Attacks: Jailbreaking Black-Box LLMs Automatically
Mehrotra, A., Zampetakis, M., Kassianik, P., Nelson, B., Anderson, H., Singer, Y., & Karbasi, A. (2023) · arXiv · doi:10.48550/arxiv.2312.02119 · Cited by 183
A Survey of AI-Generated Content (AIGC)
Cao, Y., Li, S., Liu, Y., Yan, Z., Dai, Y., Yu, P., & Sun, L. (2025) · ACM Computing Surveys · doi:10.1145/3704262 · Cited by 182
The TESCREAL bundle: Eugenics and the promise of utopia through artificial general intelligence
Gebru, T., & Torres, É. P. (2024) · First Monday · doi:10.5210/fm.v29i4.13636 · Cited by 181
Self-citationEditing Factual Knowledge in Language Models
De Cao, N., Aziz, W., & Titov, I. (2021) · Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2021.emnlp-main.522 · Cited by 180
“Technology built upon pre-trained LMs inherits some or all of their potential harms (Bender et al., 2021).”
BackgroundQuality at a Glance: An Audit of Web-Crawled Multilingual Datasets
Kreutzer, J., Caswell, I., Wang, L., Wahab, A., van Esch, D., Ulzii-Orshikh, N., Tapo, A., Subramani, N., Sokolov, A., Sikasote, C., et al. (2022) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00447 · Cited by 179
“As the scale of ML research grows, it becomes increasingly difficult to validate automatically collected and curated datasets (Biderman and Scheirer, 2020; Prabhu and Birhane, 2020; Bender et al., 2021).”
BackgroundWhat Explains Teachers’ Trust in AI in Education Across Six Countries?
Viberg, O., Cukurova, M., Feldman-Maggor, Y., Alexandron, G., Shirai, S., Kanemune, S., Wasson, B., Tømte, C., Spikol, D., Milrad, M., et al. (2025) · International Journal of Artificial Intelligence in Education · doi:10.1007/s40593-024-00433-x · Cited by 179
“Given the inherent challenges in uncertainty avoidance with these technologies (Bender et al., 2021), their integration into AI-EdTech requires careful investigation.”
Quality at a Glance: An Audit of Web-Crawled Multilingual Datasets
Kreutzer, J., Caswell, I., Wang, L., Wahab, A., van Esch, D., Ulzii-Orshikh, N., Tapo, A., Subramani, N., Sokolov, A., Sikasote, C., et al. (2021) · arXiv · doi:10.48550/arxiv.2103.12028 · Cited by 179
Toxicity in ChatGPT: Analyzing Persona-assigned Language Models
Deshpande, A., Murahari, V., Rajpurohit, T., Kalyan, A., & Narasimhan, K. (2023) · arXiv · doi:10.48550/arxiv.2304.05335 · Cited by 178
Training Compute-Optimal Large Language Models
Hoffmann, J., Borgeaud, S., Mensch, A., Buchatskaya, E., Cai, T., Rutherford, E., Casas, D. D. L., Hendricks, L. A., Welbl, J., Clark, A., et al. (2022) · Advances in Neural Information Processing Systems 35 · doi:10.52202/068431-2176 · Cited by 932
“Large Language Models carry potential risks such as outputting offensive language, propagating social biases, and leaking private information (Bender et al., 2021; Weidinger et al., 2021).”
"Kelly is a Warm Person, Joseph is a Role Model": Gender Biases in LLM-Generated Reference Letters
Wan, Y., Pu, G., Sun, J., Garimella, A., Chang, K.-W., & Peng, N. (2023) · arXiv · doi:10.48550/arxiv.2310.09219 · Cited by 175
Uses of Generative AI in the Newsroom: Mapping Journalists’ Perceptions of Perils and Possibilities
Cools, H., & Diakopoulos, N. (2026) · Journalism Practice · doi:10.1080/17512786.2024.2394558 · Cited by 174
Xu, H., Omitaomu, F., Sabri, S., Zlatanova, S., Li, X., & Song, Y. (2024) · Urban Informatics · doi:10.1007/s44212-024-00060-w · Cited by 178
“Concerns are raised regarding the factuality of AI-Generated response to domain-specific questions that require the most current and dynamic information, and specili-zed knowledge (Cao et al. 2021; Bender et al. 2021).”
Pedagogical Applications of Generative AI in Higher Education: A Systematic Review of the Field
Qian, Y. (2025) · TechTrends · doi:10.1007/s11528-025-01100-1 · Cited by 206
“The National AI Initiative (2021), a U.S. policy framework, advocates embedding AI literacy into curricula, suggesting that courses on prompt design and AI limitations could help mitigate overreliance.”
Gallent Torres, C., Zapata González, A., & Ortego Hernando, J. L. (2023) · RELIEVE - Revista Electrónica de Investigación y Evaluación Educativa · doi:10.30827/relieve.v29i2.29134 · Cited by 170
Factuality challenges in the era of large language models and opportunities for fact-checking
Augenstein, I., Baldwin, T., Cha, M., Chakraborty, T., Ciampaglia, G. L., Corney, D., DiResta, R., Ferrara, E., Hale, S., Halevy, A., et al. (2024) · Nature Machine Intelligence · doi:10.1038/s42256-024-00881-z · Cited by 187
Large language models challenge the future of higher education
Milano, S., McGrane, J. A., & Leonelli, S. (2023) · Nature Machine Intelligence · doi:10.1038/s42256-023-00644-2 · Cited by 166
AI agents and agentic systems: a multi-expert analysis
Hughes, L., Dwivedi, Y. K., Malik, T., Shawosh, M., Albashrawi, M. A., Jeon, I., Dutot, V., Appanderanda, M., Crick, T., De’, R., et al. (2025) · Journal of Computer Information Systems · doi:10.1080/08874417.2025.2483832 · Cited by 168
ED-AI Lit: An Interdisciplinary Framework for AI Literacy in Education
Allen, L. K., & Kendeou, P. (2024) · Policy Insights from the Behavioral and Brain Sciences · doi:10.1177/23727322231220339 · Cited by 168
“Prior research has emphasized issues surrounding the inter-pretability and limitations of AI systems, as well as ethical AI calls for greater transparency and accountability in AI design (Bender et al., 2021; Hutchinson & Mitchell, 2019).”
MethodLarge Language Models can Accurately Predict Searcher Preferences
Thomas, P., Spielman, S., Craswell, N., & Mitra, B. (2024) · Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval · doi:10.1145/3626772.3657707 · Cited by 168
“3 Lastly, the ecological costs of these LLMs are still heavily debated [Bender et al. 2021; Bommasani et al. 2021; Dodge et al. 2022; Patterson et al. 2022, 2021; Wu et al. 2022] but represent an important aspect in which these models should”
BackgroundJo, E., Epstein, D. A., Jung, H., & Kim, Y.-H. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581503 · Cited by 174
“Also, there exists a possibility that the unethical or biased phrases ingrained in the models’ pre-training datasets might be exposed in the model’s output, causing the chatbot to say socially biased [6, 7, 21, 67, 68] or toxic [22] messages.”
BackgroundGender Bias in Machine Translation
Savoldi, B., Gaido, M., Bentivogli, L., Negri, M., & Turchi, M. (2021) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00401 · Cited by 165
“With language technologies entering widespread use and being deployed at a massive scale, their societal impact has raised concern both within (Hovy and Spruit, 2016; Bender et al., 2021) and outside (Dastin, 2018) the scientific community.”
BackgroundMethodTransforming Conversations with AI—A Comprehensive Study of ChatGPT
Bansal, G., Chamola, V., Hussain, A., Guizani, M., & Niyato, D. (2024) · Cognitive Computation · doi:10.1007/s12559-023-10236-2 · Cited by 164
What Does it Mean for a Language Model to Preserve Privacy?
Brown, H., Lee, K., Mireshghallah, F., Shokri, R., & Tramèr, F. (2022) · 2022 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3531146.3534642 · Cited by 165
“Algorithms for learning language models (notably transformer LMs) show an unprecedented performance on extremely large models with hundreds of billions of parameters trained on extremely large datasets [7, 37, 20, 100, 45].”
BackgroundCLIMATEBERT: A Pretrained Language Model for Climate-Related Text
Webersinke, N., Kraus, M., Bingler, J., & Leippold, M. (2022) · SSRN Electronic Journal · doi:10.2139/ssrn.4229146 · Cited by 164
“This way, we try to take into account the issues mentioned by Bender et al. (2021).”
MethodSætra, H. S. (2021) · Sustainability · doi:10.3390/su13041738 · Cited by 163
“Here, the authors argued that the current trajectory of AI, with natural language models trained on increasingly large data sets, are unsustainable and harmful in a number of ways—including its massive carbon footprint [71].”
BackgroundWhittaker, M. (2021) · Interactions · doi:10.1145/3488666 · Cited by 163
“They’ve also garnered a lot of recent media attention, and been subject to sustained critique focused on significant bias, environmental costs, and concentrated power [5].”
BackgroundWhat ChatGPT Tells Us about Gender: A Cautionary Tale about Performativity and Gender Biases in AI
Gross, N. (2023) · Social Sciences · doi:10.3390/socsci12080435 · Cited by 162
“Bender et al. (2021) highlight the unfathomable amount of data that go into LLMs, and these data inevitably include stereotypical associations and derogatory portrayals of gender.”
BackgroundMethodInfluentialSison, A. J. G., Daza, M. T., Gozalo-Brizuela, R., & Garrido-Merchán, E. C. (2024) · International Journal of Human–Computer Interaction · doi:10.1080/10447318.2023.2225931 · Cited by 169
“It has also been called a “ stochastic parrot, ” randomly stitching words together without reference or meaning (Bender et al., 2021).”
Sparks: Inspiration for Science Writing using Language Models
Gero, K. I., Liu, V., & Chilton, L. (2022) · Designing Interactive Systems Conference · doi:10.1145/3532106.3533533 · Cited by 161
“There have been several calls to be more careful and explicit with the data that these pre-trained models are trained on [7, 8, 14].”
BackgroundMethodInfluentialKabir, S., Udo-Imeh, D. N., Kou, B., & Zhang, T. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642596 · Cited by 160
“Therefore, it gives an extremely verbose response that contains the definitions of encapsulation, inheritance, etc., which is not useful in terms of identifying the differences originally asked for.”
BackgroundAssessing the Fairness of AI Systems: AI Practitioners' Processes, Challenges, and Needs for Support
Madaio, M., Egede, L., Subramonyam, H., Wortman Vaughan, J., & Wallach, H. (2022) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3512899 · Cited by 156
Symbolic Knowledge Distillation: from General Language Models to Commonsense Models
West, P., Bhagavatula, C., Hessel, J., Hwang, J., Jiang, L., Le Bras, R., Lu, X., Welleck, S., & Choi, Y. (2022) · Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/2022.naacl-main.341 · Cited by 156
“Recent work (Bender et al., 2021) has highlighted the risks of models trained on massive text resources, as GPT-3 (Brown et al., 2020) is, which we use for generation.”
Background
Metadata
- Authors
- Bender, E. M., Gebru, T., McMillan-Major, A., & Shmitchell, S.
- Affiliations
- University of Washington (US)
- Black in AI, Palo Alto, CA, USA
- The Aether
- Venue
- Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency
- Publisher
- ACM
- Published
- 2021-03
- Pages
- 610-623
- OpenAlex
- W3133702157
- Cited by
- 6,988
- References
- 117
- Citations by year
- 2026: 2484 · 2025: 1887 · 2024: 1153 · 2023: 842 · 2022: 384 · 2021: 229
- FWCI
- 300.72
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- ACM Conference on Fairness, Accountability, and Transparency (FAccT) (publishedVersion, cc-by)
- Creighton Digital Repository (CDR) (submittedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Keywords
- large language models, model size, AI risks, financial costs, data curation, dataset documentation, stakeholder value, BERT, GPT, fine-tuning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.