Taxonomy of Risks posed by Language Models
Open accessThe DOI matches its Crossref record; 149 of the 149 references it cites were found in a registry or an index.
Abstract
Responsible innovation on large-scale Language Models (LMs) requires foresight into and in-depth understanding of the risks these models may pose. This paper develops a comprehensive taxonomy of ethical and social risks associated with LMs. We identify twenty-one risks, drawing on expertise and literature from computer science, linguistics, and the social sciences. We situate these risks in our taxonomy of six risk areas: I. Discrimination, Hate speech and Exclusion, II. Information Hazards, III. Misinformation Harms, IV. Malicious Uses, V. Human-Computer Interaction Harms, and VI. Environmental and Socioeconomic harms. For risks that have already been observed in LMs, the causal mechanism leading to harm, evidence of the risk, and approaches to risk mitigation are discussed. We further describe and analyse risks that have not yet been observed but are anticipated based on assessments of other language technologies, and situate these in the same taxonomy. We underscore that it is the responsibility of organizations to engage with the mitigations we discuss throughout the paper. We close by highlighting challenges and directions for further research on risk evaluation and mitigation with the goal of ensuring that language models are developed responsibly.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Crucially, safety guardrails are typically set in place in line with developers ' judgments about what constitutes risk or harm, such as hate speech, toxicity, or election interference (Bommasani et al., 2021; Weidinger et al., 2022).”
From "Just Asking Questions": Doing Our Own Research on Conspiratorial Ideation by Generative AI Chatbots (2026) · cites this paper as Weidinger et al., 2022
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“This means that patterns of bias, misinformation, and conspiracist discourse present in the training data can be reproduced and potentially amplified by the model outputs, unless safety guardrails are put in place (Bender et al., 2021; Weidinger et al., 2022).”
From "Just Asking Questions": Doing Our Own Research on Conspiratorial Ideation by Generative AI Chatbots (2026) · cites this paper as Weidinger et al., 2022
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The widespread availability of powerful language models will also significantly reduce the costs associated with content generation and lower the technical proficiency required to conduct such operations (Brundage et al., 2018;Weidinger et al., 2022).”
From Anatomy of an AI-powered malicious social botnet (2024) · cites this paper as Weidinger et al., 2022
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The rapid advancement of generative AI, marked by accelerated software and hardware innovation and a proliferation of novel applications, has been accompanied by growing societal concerns and numerous instances of misuse (Bender et al. 2021;Weidinger et al. 2022;Bird, Ungless, and Kasirzadeh 2023).”
From Epistemic Injustice in Generative AI (2024) · cites this paper as Weidinger et al. 2022
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The ability to probe and shape LLM personality traits is pertinent to the open problem of responsible AI alignment54 and harm mitigation55.”
From A psychometric framework for evaluating and shaping personality traits in large language models (2025) · cites this paper as [55]
Experimental — accuracy not yet measured
3 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372 · cited 6×
“Current state of the art LMs are primarily trained in English or Mandarin Chinese [30, 37, 54, 148, 160] and perform better in these compared to any other languages [196].”
BackgroundInfluentialDiscipline and Punish: The Birth of the Prison.
Zukin, S., Foucault, M., & Sheridan, A. (1996) · Contemporary Sociology · doi:10.2307/2077073 · Cited by 15,361
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., & Liu, P. J. (2019) · arXiv · doi:10.48550/arxiv.1910.10683 · Cited by 10,929
On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜
Bender, E. M., Gebru, T., McMillan-Major, A., & Shmitchell, S. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445922 · Cited by 6,988
Deep Learning with Differential Privacy
Abadi, M., Chu, A., Goodfellow, I., McMahan, H. B., Mironov, I., Talwar, K., & Zhang, L. (2016) · Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security · doi:10.1145/2976749.2978318 · Cited by 6,578 · cited 1×
“One approach to preventing privacy leaks is to apply algorithmic tools such as differential privacy methods during LM training [1, 153].”
MethodExplanation in artificial intelligence: Insights from the social sciences
Miller, T. (2019) · Artificial Intelligence · doi:10.1016/j.artint.2018.07.007 · Cited by 5,302
Energy and Policy Considerations for Deep Learning in NLP
Strubell, E., Ganesh, A., & McCallum, A. (2019) · arXiv · doi:10.48550/arxiv.1906.02243 · Cited by 3,735
Energy and Policy Considerations for Deep Learning in NLP
Strubell, E., Ganesh, A., & McCallum, A. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1355 · Cited by 3,684 · cited 1×
“LMs and other large machine learning models create significant energy demands during training and operation [15, 148, 176], and correspondingly high carbon emissions”
BackgroundDense passage retrieval for open-domain question answering
Karpukhin, V., Oguz, B., Min, S., Lewis, P., Wu, L., Edunov, S., Chen, D., & Yih, W.-T. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.550 · Cited by 3,575
Developing a framework for responsible innovation
Stilgoe, J., Owen, R., & Macnaghten, P. (2013) · Research Policy · doi:10.1016/j.respol.2013.05.008 · Cited by 3,184 · cited 2×
“multiple perspectives and communities and then acting on these insights [175].”
BackgroundBarocas, S., & Selbst, A. D. (2014) · doi:10.2139/ssrn.2477899 · Cited by 3,048
Semantics derived automatically from language corpora contain human-like biases
Caliskan, A., Bryson, J. J., & Narayanan, A. (2017) · Science · doi:10.1126/science.aal4230 · Cited by 2,887 · cited 1×
“The reproduction of harmful stereotypes is well-documented in models that represent natural language [32].”
BackgroundPrivate traits and attributes are predictable from digital records of human behavior
Kosinski, M., Stillwell, D., & Graepel, T. (2013) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1218772110 · Cited by 2,829 · cited 1×
“Leveraging language processing tools and large public datasets to infer protected and other personal traits is an active area of research [107, 140, 146, 200], despite serious ethical concerns [6, 185].”
BackgroundOn the Opportunities and Risks of Foundation Models
Bommasani, R., Hudson, D. A., Adeli, E., Altman, R., Arora, S., von Arx, S., Bernstein, M. S., Bohg, J., Bosselut, A., Brunskill, E., et al. (2021) · arXiv · doi:10.48550/arxiv.2108.07258 · Cited by 2,676
PaLM: Scaling Language Modeling with Pathways
Chowdhery, A., Narang, S., Devlin, J., Bosma, M., Mishra, G., Roberts, A., Barham, P., Chung, H. W., Sutton, C., Gehrmann, S., et al. (2022) · arXiv · doi:10.48550/arxiv.2204.02311 · Cited by 2,590 · cited 1×
“Recent advancements in LM research [30, 38, 75, 170, 205] are rooted in the capacity to increase LM size in terms of number of”
BackgroundEquality of Opportunity in Supervised Learning
Hardt, M., Price, E., & Srebro, N. (2016) · arXiv · doi:10.48550/arxiv.1610.02413 · Cited by 2,536
Zero-shot text-to-image generation
Ramesh, A., Pavlov, M., Goh, G., Gray, S., Voss, C., Radford, A., Chen, M., & Sutskever, I. (2021) · arXiv · doi:10.48550/arxiv.2102.12092 · Cited by 2,417 · cited 1×
“Lastly, this paper does not explicitly discuss risks that depend on multiple modalities, for example from models that combine language with other domains such as vision or robotics [44, 154], though several of the insights in this paper are translatable to such models.”
BackgroundXAI—Explainable artificial intelligence
Gunning, D., Stefik, M., Choi, J., Miller, T., Stumpf, S., & Yang, G.-Z. (2019) · Science Robotics · doi:10.1126/scirobotics.aay7120 · Cited by 2,053 · cited 1×
“Moreover, explainability and interpretability research is needed as groundwork to measure LM fairness [24, 66, 129].”
BackgroundSuperintelligence: paths, dangers, strategies
(2015) · Choice Reviews Online · doi:10.5860/choice.187536 · Cited by 1,979
TinyBERT: Distilling BERT for natural language understanding
Jiao, X., Yin, Y., Shang, L., Jiang, X., Chen, X., Li, L., Wang, F., & Liu, Q. (2020) · Findings of the Association for Computational Linguistics: EMNLP 2020 · doi:10.18653/v1/2020.findings-emnlp.372 · Cited by 1,786
Schwartz, R., Dodge, J., Smith, N. A., & Etzioni, O. (2020) · Communications of the ACM · doi:10.1145/3381831 · Cited by 1,647
Review of “Race After Technology: Abolitionist Tools for the New Jim Code”
Jipguep-Akhtar, M. (2020) · Social Forces · doi:10.1093/sf/soz162 · Cited by 1,473 · cited 1×
“Unfair treatment can also emerge from LMs that perform better for some social groups than others [18].”
BackgroundEvaluating Large Language Models Trained on Code
Chen, M., Tworek, J., Jun, H., Yuan, Q., Pinto, H. P. D. O., Kaplan, J., Edwards, H., Burda, Y., Joseph, N., Brockman, G., et al. (2021) · arXiv · doi:10.48550/arxiv.2107.03374 · Cited by 1,436 · cited 4×
“Creators of the assistive coding tool Co-Pilot based on GPT-3 suggest that such tools may lower the cost of developing polymorphic malware which is able to change its features in order to evade detection [37].”
BackgroundInfluentialRace and Epistemologies of Ignorance
(2007) · State University of New York Press eBooks · doi:10.1353/book5200 · Cited by 1,355
The Curious Case of Neural Text Degeneration
Holtzman, A., Buys, J., Du, L., Forbes, M., & Choi, Y. (2019) · arXiv · doi:10.48550/arxiv.1904.09751 · Cited by 1,304 · cited 1×
“Instead, methods that focus on the most likely utterances – while introducing a small amount of variability (e.g. beam search and nucleus sampling) – have been found to produce better results in practice [78]).”
BackgroundA Survey on Automatic Detection of Hate Speech in Text
Fortuna, P., & Nunes, S. (2019) · ACM Computing Surveys · doi:10.1145/3232676 · Cited by 1,241 · cited 2×
“…create a range of harms, such as promoting social stereotypes that perpetuate the derogatory representation or unfair treat-ment of marginalised groups [22], inciting hate or violence [57], causing profound offence [199], or reinforcing social norms that exclude or marginalise identities [15, 58].”
Background(2022) · bepress Legal Repository · Cited by 1,139
Leveraging Passage Retrieval with Generative Models for Open Domain Question Answering
Izacard, G., & Grave, E. (2021) · Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume · doi:10.18653/v1/2021.eacl-main.74 · Cited by 1,109
Gorwa, R., Binns, R., & Katzenbach, C. (2020) · Big Data & Society · doi:10.1177/2053951719897945 · Cited by 1,097 · cited 1×
“LMs may generate language that includes profanities, identity attacks, insults, threats, language that incites violence, or language that causes justified of-fence - as such language is prominent online [57, 64, 143, 191].”
BackgroundComputer-based personality judgments are more accurate than those made by humans
Youyou, W., Kosinski, M., & Stillwell, D. (2015) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1418680112 · Cited by 1,086 · cited 1×
“Leveraging language processing tools and large public datasets to infer protected and other personal traits is an active area of research [107, 140, 146, 200], despite serious ethical concerns [6, 185].”
Background(2019) · Cited by 1,082
Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity
Fedus, W., Zoph, B., & Shazeer, N. (2021) · arXiv · doi:10.48550/arxiv.2101.03961 · Cited by 1,005 · cited 1×
“Current state of the art LMs are primarily trained in English or Mandarin Chinese [30, 37, 54, 148, 160] and perform better in these compared to any other languages [196].”
BackgroundArtificial Intelligence, Automation and Work
Acemoglu, D., & Restrepo, P. (2018) · doi:10.3386/w24196 · Cited by 962
Training Compute-Optimal Large Language Models
Hoffmann, J., Borgeaud, S., Mensch, A., Buchatskaya, E., Cai, T., Rutherford, E., Casas, D. D. L., Hendricks, L. A., Welbl, J., Clark, A., et al. (2022) · arXiv · doi:10.48550/arxiv.2203.15556 · Cited by 929 · cited 1×
“Recent advancements in LM research [30, 38, 75, 170, 205] are rooted in the capacity to increase LM size in terms of number of”
BackgroundSelf-citationTruthfulQA: Measuring How Models Mimic Human Falsehoods
Lin, S., Hilton, J., & Evans, O. (2021) · arXiv · doi:10.48550/arxiv.2109.07958 · Cited by 899 · cited 2×
“are prompted regarding common misconceptions [118].”
BackgroundMegatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism
Shoeybi, M., Patwary, M., Puri, R., LeGresley, P., Casper, J., & Catanzaro, B. (2019) · arXiv · doi:10.48550/arxiv.1909.08053 · Cited by 893 · cited 2×
“Recent advancements in LM research [30, 38, 75, 170, 205] are rooted in the capacity to increase LM size in terms of number of parameters and size of training data [15].”
BackgroundProceedings of the 2019 CHI Conference on Human Factors in Computing Systems
(2019) · doi:10.1145/3290605 · Cited by 841
The Risk of Racial Bias in Hate Speech Detection
Sap, M., Card, D., Gabriel, S., Choi, Y., & Smith, N. A. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1163 · Cited by 817 · cited 1×
“showing that more work is needed to address discrimination and hate speech risks in tandem [49, 103, 165, 194].”
BackgroundAutomatic personality assessment through social media language.
Park, G., Schwartz, H. A., Eichstaedt, J. C., Kern, M. L., Kosinski, M., Stillwell, D. J., Ungar, L. H., & Seligman, M. E. P. (2015) · Journal of Personality and Social Psychology · doi:10.1037/pspp0000020 · Cited by 802 · cited 1×
“Leveraging language processing tools and large public datasets to infer protected and other personal traits is an active area of research [107, 140, 146, 200], despite serious ethical concerns [6, 185].”
BackgroundRising Income Inequality: Technology, or Trade and Financial Globalization?
Jaumotte, F., Lall, S., & Papageorgiou, C. (2013) · IMF Economic Review · doi:10.1057/imfer.2013.7 · Cited by 777 · cited 1×
“reflects a general economic trend whereby the single biggest driver of increasing global income inequality is technological progress [91].”
Improving Fairness in Machine Learning Systems: What Do Industry Practitioners Need?
Holstein, K., Wortman Vaughan, J., Daumé, H., Dudik, M., & Wallach, H. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300830 · Cited by 777
Wang, Y., & Kosinski, M. (2018) · Journal of Personality and Social Psychology · doi:10.1037/pspa0000098 · Cited by 766 · cited 1×
“Such systems may never be accurate - for example, image classification models that attempt to infer unobservable characteristics, such as sexual orientation from a portrait [190], are inherently prone to error.”
BackgroundThe State and Fate of Linguistic Diversity and Inclusion in the NLP World
Joshi, P., Santy, S., Budhiraja, A., Bali, K., & Choudhury, M. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.560 · Cited by 766
Extracting Training Data from Large Language Models
Carlini, N., Tramer, F., Wallace, E., Jagielski, M., Herbert-Voss, A., Lee, K., Roberts, A., Brown, T., Song, D., Erlingsson, U., et al. (2020) · arXiv · doi:10.48550/arxiv.2012.07805 · Cited by 727 · cited 3×
“Observed risks include privacy violations [34].”
BackgroundInfluentialMiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformers
Wang, W., Wei, F., Dong, L., Bao, H., Yang, N., & Zhou, M. (2020) · arXiv · doi:10.48550/arxiv.2002.10957 · Cited by 703 · cited 1×
“ciency gains during training and inference [116], for example via pruning [164], distillation [93, 189], or fine-tuning [148].”
BackgroundRealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Models
Gehman, S., Gururangan, S., Sap, M., Choi, Y., & Smith, N. A. (2020) · Findings of the Association for Computational Linguistics: EMNLP 2020 · doi:10.18653/v1/2020.findings-emnlp.301 · Cited by 693
Measuring and Mitigating Unintended Bias in Text Classification
Dixon, L., Li, J., Sorensen, J., Thain, N., & Vasserman, L. (2018) · Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3278721.3278729 · Cited by 674
StereoSet: Measuring stereotypical bias in pretrained language models
Nadeem, M., Bethke, A., & Reddy, S. (2020) · arXiv · doi:10.48550/arxiv.2004.09456 · Cited by 643 · cited 1×
“The StereoSet benchmark finds that certain LMs exhibit strong stereotypical associations on race, gender, religion, and profession [133].”
BackgroundTowards a Human-like Open-Domain Chatbot
Adiwardana, D., Luong, M.-T., So, D. R., Hall, J., Fiedel, N., Thoppilan, R., Yang, Z., Kulshreshtha, A., Nemade, G., Lu, Y., et al. (2020) · arXiv · doi:10.48550/arxiv.2001.09977 · Cited by 614 · cited 2×
“Such research can draw on methods from human-computer-interaction research [5].”
BackgroundMethodAnthropomorphism: Opportunities and Challenges in Human–Robot Interaction
Złotowski, J., Proudfoot, D., Yogeeswaran, K., & Bartneck, C. (2015) · International Journal of Social Robotics · doi:10.1007/s12369-014-0267-6 · Cited by 598
mT5: A massively multilingual pre-trained text-to-text transformer
Xue, L., Constant, N., Roberts, A., Kale, M., Al‐Rfou, R., Siddhant, A., Barua, A., & Raffel, C. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.11934 · Cited by 594 · cited 1×
“Efforts to improve language performance include better representation of different languages in training corpora [25, 182, 198].”
BackgroundDefending Against Neural Fake News
Zellers, R., Holtzman, A., Rashkin, H., Bisk, Y., Farhadi, A., Roesner, F., & Choi, Y. (2019) · arXiv · doi:10.48550/arxiv.1905.12616 · Cited by 569 · cited 2×
“Here, the generative model itself may be most effective in detecting synthetic text from itself: one paper found that ’counterintuitively, the best defence against Grover the LM was Grover itself’ [203].”
BackgroundThe Impact of Artificial Intelligence on the Labor Market
Webb, M. (2019) · SSRN Electronic Journal · doi:10.2139/ssrn.3482150 · Cited by 531
Aligning AI With Shared Human Values
Hendrycks, D., Burns, C., Basart, S., Critch, A., Li, J., Song, D., & Steinhardt, J. (2020) · arXiv · doi:10.48550/arxiv.2008.02275 · Cited by 517 · cited 1×
“Several LMs failed to reliably distinguish between ethical or unethical actions, indicating they may advise unethical behaviours [72].”
BackgroundCalibrate Before Use: Improving Few-Shot Performance of Language Models
Zhao, T. Z., Wallace, E., Feng, S., Klein, D., & Singh, S. (2021) · arXiv · doi:10.48550/arxiv.2102.09690 · Cited by 513 · cited 2×
“For example, GPT-3 was found to ‘often predict common entities such as “America” when the ground-truth answer is instead a rare entity in the training data’, such as Keetmansoop, Namibia [206].”
BackgroundPlug and Play Language Models: A Simple Approach to Controlled Text Generation
Dathathri, S., Madotto, A., Lan, J., Hung, J., Frank, E., Molino, P., Yosinski, J., & Liu, R. (2019) · arXiv · doi:10.48550/arxiv.1912.02164 · Cited by 508 · cited 2×
“Though the standard LM training objective encourages LMs to mirror language found in the training data, generated language can be further constrained and steered by additional objectives during training [90, 207] or specific sampling techniques [42].”
BackgroundCarbon Emissions and Large Neural Network Training
Patterson, D., Gonzalez, J., Le, Q., Liang, C., Munguia, L.-M., Rothchild, D., So, D., Texier, M., & Dean, J. (2021) · arXiv · doi:10.48550/arxiv.2104.10350 · Cited by 486
Our Twitter Profiles, Our Selves: Predicting Personality with Twitter
Quercia, D., Kosinski, M., Stillwell, D., & Crowcroft, J. (2011) · 2011 IEEE Third Int'l Conference on Privacy, Security, Risk and Trust and 2011 IEEE Third Int'l Conference on Social Computing · doi:10.1109/passat/socialcom.2011.26 · Cited by 478 · cited 1×
“Leveraging language processing tools and large public datasets to infer protected and other personal traits is an active area of research [107, 140, 146, 200], despite serious ethical concerns [6, 185].”
BackgroundEthical and social risks of harm from Language Models
Weidinger, L., Mellor, J., Rauh, M., Griffin, C., Uesato, J., Huang, P.-S., Cheng, M., Glaese, M., Balle, B., Kasirzadeh, A., et al. (2021) · arXiv · doi:10.48550/arxiv.2112.04359 · Cited by 477 · cited 1×
“including who gets to set these thresholds [193].”
BackgroundSelf-citationPersistent Anti-Muslim Bias in Large Language Models
Abid, A., Farooqi, M., & Zou, J. (2021) · Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3461702.3462624 · Cited by 476 · cited 1×
“in 23% of test cases [2]; and gender bias, presenting fictional female characters as more domestic than male counterparts [120].”
BackgroundFine-Tuning Language Models from Human Preferences
Ziegler, D. M., Stiennon, N., Wu, J., Brown, T. B., Radford, A., Amodei, D., Christiano, P., & Irving, G. (2019) · arXiv · doi:10.48550/arxiv.1909.08593 · Cited by 413 · cited 1×
“Though the standard LM training objective encourages LMs to mirror language found in the training data, generated language can be further constrained and steered by additional objectives during training [90, 207] or specific sampling techniques [42].”
BackgroundSelf-citationImproving language models by retrieving from trillions of tokens
Borgeaud, S., Mensch, A., Hoffmann, J., Cai, T., Rutherford, E., Millican, K., Driessche, G. V. D., Lespiau, J.-B., Damoc, B., Clark, A., et al. (2021) · arXiv · doi:10.48550/arxiv.2112.04426 · Cited by 382 · cited 4×
“Technical approaches to reducing risks of environmental harm include segmenting LMs into less large LMs that search and retrieve information from a distinct data corpus [25, 88, 97, 102, 115].”
BackgroundInfluentialSelf-citationMiner, A. S., Milstein, A., Schueller, S., Hegde, R., Mangurian, C., & Linos, E. (2016) · JAMA Internal Medicine · doi:10.1001/jamainternmed.2016.0400 · Cited by 368 · cited 1×
“For example, misin-formation on medical dosages may lead a user to cause harm to themselves [21, 130].”
BackgroundInternet-Augmented Dialogue GenerationPreprint
Komeili, M., Shuster, K., & Weston, J. (2021) · Annual Meeting of the Association for Computational Linguistics · doi:10.48550/arxiv.2107.07566 · Cited by 334 · cited 1×
“In additionto increasing representation of marginalised groups and addressing specific needs in downstream applications [159], approaches to expanding or up-dating LMs in real-time also include modes that continue to learn online, as changes occur to a training corpus [8, 43, 106, 112, 179].”
BackgroundThe social impact of natural language processing
Hovy, D., & Spruit, S. L. (2016) · Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) · doi:10.18653/v1/p16-2096 · Cited by 329 · cited 1×
“Choices on model architecture may also have an impact, as it has been proposed that the architecture of current LMs and particularly tokenisation is well-suited to English, but not morphologically more complex languages [14, 79, 162].”
BackgroundInfant-like Social Interactions between a Robot and a Human Caregiver
Breazeal, C., & Scassellati, B. (2000) · Adaptive Behavior · doi:10.1177/105971230000800104 · Cited by 327 · cited 1×
“Similarly, in other interactive technologies it was found that the more human-like a system appears, the more likely it is that users attribute more human traits and capabilities to that system [29, 126, 208].”
BackgroundBickmore, T. W., Trinh, H., Olafsson, S., O'Leary, T. K., Asadi, R., Rickles, N. M., & Cruz, R. (2018) · Journal of Medical Internet Research · doi:10.2196/11510 · Cited by 308 · cited 1×
“For example, misin-formation on medical dosages may lead a user to cause harm to themselves [21, 130].”
BackgroundDemographic dialectal variation in social media: a case study of African-American English
Blodgett, S. L., Green, L., & O'Connor, B. (2016) · Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d16-1120 · Cited by 291 · cited 1×
“Disparate performance can also occur based on slang, dialect, sociolect, and other aspects that vary within a single language [23].”
BackgroundBeyond Binary Labels: Political Ideology Prediction of Twitter Users
Preoţiuc-Pietro, D., Liu, Y., Hopkins, D., & Ungar, L. (2017) · Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p17-1068 · Cited by 289 · cited 1×
“Tweets) are already being analysed to predict private information such as political orientation [121, 144], age [131, 135], and health data such as addiction relapses [63].”
BackgroundGender and Representation Bias in GPT-3 Generated Stories
Lucy, L., & Bamman, D. (2021) · Proceedings of the Third Workshop on Narrative Understanding · doi:10.18653/v1/2021.nuse-1.5 · Cited by 287 · cited 1×
“GPT-3 [30] exhibited bias based on religion, analogising ‘“Muslim” to “terrorist” in 23% of test cases [2]; and gender bias, presenting fictional female characters as more domestic than male counterparts [120].”
BackgroundGeneralization through Memorization: Nearest Neighbor Language Models
Khandelwal, U., Levy, O., Jurafsky, D., Zettlemoyer, L., & Lewis, M. (2019) · arXiv · doi:10.48550/arxiv.1911.00172 · Cited by 285 · cited 2×
“Technical approaches to reducing risks of environmental harm include segmenting LMs into less large LMs that search and retrieve information from a distinct data corpus [25, 88, 97, 102, 115].”
BackgroundCave, S., & Dihal, K. (2020) · Philosophy & Technology · doi:10.1007/s13347-020-00415-6 · Cited by 279 · cited 1×
“Non-linguistic AI systems were found to typically present as ‘intelligent, professional, or powerful’ and as ethnically White, reinforcing historical racist associations between intelligence and whiteness [35].”
BackgroundHutchinson, B., Smart, A., Hanna, A., Denton, R., Greer, C., Kjartansson, O., Barnes, P., & Mitchell, M. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445918 · Cited by 279
WebGPT: Browser-assisted question-answering with human feedback
Nakano, R., Hilton, J., Balaji, S., Wu, J., Ouyang, L., Kim, C., Hesse, C., Jain, S., Kosaraju, V., Saunders, W., et al. (2021) · arXiv · doi:10.48550/arxiv.2112.09332 · Cited by 272 · cited 3×
“[134] improved performance of their model WebGPT which”
BackgroundOgnyanova, K., Lazer, D., Robertson, R. E., & Wilson, C. (2020) · Harvard Kennedy School Misinformation Review · doi:10.37016/mr-2020-024 · Cited by 261
Chatbot advertising effectiveness: When does the message get through?
Van den Broeck, E., Zarouali, B., & Poels, K. (2019) · Computers in Human Behavior · doi:10.1016/j.chb.2019.04.009 · Cited by 257 · cited 1×
“In customer service chatbots, users more often accepted “intrusiveness” from chatbots that were perceived to be more helpful and useful [183], suggesting that”
BackgroundLessons from archives: strategies for collecting sociocultural data in machine learning
Jo, E. S., & Gebru, T. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372829 · Cited by 240
Scaling Language Models: Methods, Analysis & Insights from Training Gopher
Rae, J. W., Borgeaud, S., Cai, T., Millican, K., Hoffmann, J., Song, F., Aslanides, J., Henderson, S., Ring, R., Young, S., et al. (2021) · arXiv · doi:10.48550/arxiv.2112.11446 · Cited by 235 · cited 10×
“Current state of the art LMs are primarily trained in English or Mandarin Chinese [30, 37, 54, 148, 160] and perform better in these compared to any other languages [196].”
BackgroundInfluentialSelf-citationRe-imagining Algorithmic Fairness in India and Beyond
Sambasivan, N., Arnesen, E., Hutchinson, B., Doshi, T., & Prabhakaran, V. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445896 · Cited by 228 · cited 1×
“The axes along which unfair bias is encoded in the LM can be rooted in localised social hierarchies such as the Hindu caste system, making it harder to anticipate harmful social stereotypes across contexts [163].”
BackgroundMytton, D. (2021) · npj Clean Water · doi:10.1038/s41545-021-00101-w · Cited by 225
Movement pruning: Adaptive sparsity by fine-tuning
Sanh, V., Wolf, T., & Rush, A. M. (2020) · arXiv · doi:10.48550/arxiv.2005.07683 · Cited by 218 · cited 1×
“ciency gains during training and inference [116], for example via pruning [164], distillation [93, 189], or fine-tuning [148].”
BackgroundGeDi: Generative Discriminator Guided Sequence GenerationPreprint
Krause, B., Gotmare, A. D., McCann, B., Keskar, N. S., Joty, S., Socher, R., & Rajani, N. F. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2009.06367 · Cited by 202 · cited 1×
“Mitigation strategies include filtering out toxic statements from training corpora, either during initial training [194], fine-tuning after pretraining [61], filtering LM outputs [194, 197], decoding techniques [42, 108, 166] or prompt design [9, 148].”
BackgroundDouglas, D. M. (2016) · Ethics and Information Technology · doi:10.1007/s10676-016-9406-0 · Cited by 191 · cited 1×
“Disclosure of private information can have the same effects as doxing (the publication of private or identifying information about an individual with malicious intent), causing psychological and material harm [51, 119, 181].”
BackgroundRaji, I. D., Smart, A., White, R. N., Mitchell, M., Gebru, T., Hutchinson, B., Smith-Loud, J., Theron, D., & Barnes, P. (2020) · arXiv · doi:10.48550/arxiv.2001.00973 · Cited by 179 · cited 1×
“ing the methodological toolkit for LM analysis beyond benchmarks [151, 152, 180].”
BackgroundTeraPipe: Token-Level Pipeline Parallelism for Training Large-Scale Language ModelsPreprint
Li, Z., Zhuang, S., Guo, S., Zhuo, D., Zhang, H., Song, D. X., & Stoica, I. G. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2102.07988 · Cited by 166 · cited 2×
“However LM fine-tuning with differential privacy2 has been limited to small models [116, 201] and it remains to be established whether it is suitable for training LMs from scratch on large datasets of web text [188].”
BackgroundLoose tweets: an analysis of privacy leaks on twitter
Mao, H., Shuai, X., & Kapadia, A. (2011) · Proceedings of the 10th annual ACM workshop on Privacy in the electronic society · doi:10.1145/2046556.2046558 · Cited by 164
Privacy Concerns in Chatbot Interactions
Ischen, C., Araujo, T., Voorveld, H., van Noort, G., & Smit, E. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-39540-7_3 · Cited by 152
Bengio, Y. (2008) · Scholarpedia · doi:10.4249/scholarpedia.3881 · Cited by 148 · cited 1×
“LMs aim to capture statistical properties of the language present in their training corpus and can be used to make probabilistic predictions regarding sequences of tokens [17].”
On Achieving and Evaluating Language-Independence in NLP
Bender, E. M. (2011) · Linguistic Issues in Language Technology · doi:10.33011/lilt.v6i.1239 · Cited by 146
A General Language Assistant as a Laboratory for Alignment
Askell, A., Bai, Y., Chen, A., Drain, D., Ganguli, D., Henighan, T., Jones, A., Joseph, N., Mann, B., DasSarma, N., et al. (2021) · arXiv · doi:10.48550/arxiv.2112.00861 · Cited by 146 · cited 1×
“Mitigation strategies include filtering out toxic statements from training corpora, either during initial training [194], fine-tuning after pretraining [61], filtering LM outputs [194, 197], decoding techniques [42, 108, 166] or prompt design [9, 148].”
BackgroundQuestion and Answer Test-Train Overlap in Open-Domain Question Answering Datasets
Lewis, P., Stenetorp, P., & Riedel, S. (2021) · Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume · doi:10.18653/v1/2021.eacl-main.86 · Cited by 145
Understanding the Capabilities, Limitations, and Societal Impact of Large Language Models
Tamkin, A., Brundage, M., Clark, J., & Ganguli, D. (2021) · arXiv · doi:10.48550/arxiv.2102.02503 · Cited by 135 · cited 4×
“As one paper concluded, it is difficult to scope all possible (mis-)uses of LMs [180].”
BackgroundInfluentialTinyBERT: Distilling BERT for Natural Language UnderstandingPreprint
Jiao, X., Yin, Y., Shang, L., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.10351 · Cited by 133 · cited 1×
“ciency gains during training and inference [116], for example via pruning [164], distillation [93, 189], or fine-tuning [148].”
BackgroundAI and the Everything in the Whole Wide World Benchmark
Raji, I. D., Bender, E. M., Paullada, A., Denton, E., & Hanna, A. (2021) · arXiv · doi:10.48550/arxiv.2111.15366 · Cited by 127 · cited 1×
“ing the methodological toolkit for LM analysis beyond benchmarks [151, 152, 180].”
BackgroundWay Off-Policy Batch Deep Reinforcement Learning of Implicit Human Preferences in Dialog
Jaques, N., Ghandeharioun, A., Shen, J. H., Ferguson, C., Lapedriza, A., Jones, N., Gu, S., & Picard, R. (2019) · arXiv · doi:10.48550/arxiv.1907.00456 · Cited by 126 · cited 1×
“Though the standard LM training objective encourages LMs to mirror language found in the training data, generated language can be further constrained and steered by additional objectives during training [90, 207] or specific sampling techniques [42].”
BackgroundThe Importance of Modeling Social Factors of Language: Theory and Practice
Hovy, D., & Yang, D. (2021) · Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/2021.naacl-main.49 · Cited by 122 · cited 1×
“ther complicated by the context dependency of what constitutes profoundly offensive speech [80, 92, 105].”
BackgroundProfound Offense and Cultural Appropriation
Young, J. O. (2005) · The Journal of Aesthetics and Art Criticism · doi:10.1111/j.0021-8529.2005.00190.x · Cited by 118 · cited 1×
“…create a range of harms, such as promoting social stereotypes that perpetuate the derogatory representation or unfair treat-ment of marginalised groups [22], inciting hate or violence [57], causing profound offence [199], or reinforcing social norms that exclude or marginalise identities [15, 58].”
BackgroundVredenburgh, K. (2022) · Journal of Political Philosophy · doi:10.1111/jopp.12262 · Cited by 112
Challenges in Detoxifying Language Models
Welbl, J., Glaese, A., Uesato, J., Dathathri, S., Mellor, J., Hendricks, L. A., Anderson, K., Kohli, P., Coppin, B., & Huang, P.-S. (2021) · Findings of the Association for Computational Linguistics: EMNLP 2021 · doi:10.18653/v1/2021.findings-emnlp.210 · Cited by 112
Self-citationPredicting political preference of Twitter users
Makazhanov, A., Rafiei, D., & Waqar, M. (2014) · Social Network Analysis and Mining · doi:10.1007/s13278-014-0193-5 · Cited by 110 · cited 1×
“Language utterances (e.g. Tweets) are already being analysed to predict private information such as political orientation [121, 144], age [131, 135], and health data such as addiction relapses [63].”
BackgroundLAMOL: LAnguage MOdeling for Lifelong Language Learning
Sun, F.-K., Ho, C.-H., & Lee, H.-Y. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.03329 · Cited by 105 · cited 1×
“In addition to increasing representation ofmarginalised groups and addressing specific needs in downstream applications [159], approaches to expanding or updating LMs in real-time also include modes that continue to learn online, as changes occur to a training corpus [8, 43, 106, 112, 179].”
BackgroundOffensive, aggressive, and hate speech analysis: From data-centric to human-centered approach
Kocoń, J., Figas, A., Gruza, M., Puchalska, D., Kajdanowicz, T., & Kazienko, P. (2021) · Information Processing & Management · doi:10.1016/j.ipm.2021.102643 · Cited by 103 · cited 1×
“Mitigation is further complicated by the context dependency of what constitutes profoundly offensive speech [80, 92, 105].”
BackgroundPride and Prejudiced: On the Reclamation of Slurs
Jeshion, R. (2020) · Grazer Philosophische Studien · doi:10.1163/18756735-09701007 · Cited by 102
Differentially Private Fine-tuning of Language Models
Yu, D., Naik, S., Backurs, A., Gopi, S., Inan, H. A., Kamath, G., Kulkarni, J., Lee, Y. T., Manoel, A., Wutschitz, L., et al. (2021) · arXiv · doi:10.48550/arxiv.2110.06500 · Cited by 101
Bringing the People Back In: Contesting Benchmark Machine Learning Datasets
Denton, R., Hanna, A., Amironesei, R., Smart, A., Nicole, H., & Scheuerman, M. K. (2020) · arXiv · doi:10.48550/arxiv.2007.07399 · Cited by 100 · cited 1×
“Curating training data can also help to make LMs fairer [45, 84, 94].”
BackgroundHONEST: Measuring Hurtful Sentence Completion in Language Models
Nozza, D., Bianchi, F., & Hovy, D. (2021) · Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/2021.naacl-main.191 · Cited by 95
Mind the Gap: Assessing Temporal Generalization in Neural Language Models
Lazaridou, A., Kuncoro, A., Gribovskaya, E., Agrawal, D., Liska, A., Terzi, T., Gimenez, M., d'Autume, C. D. M., Kocisky, T., Ruder, S., et al. (2021) · arXiv · doi:10.48550/arxiv.2102.01951 · Cited by 90 · cited 2×
“In addition to increasing representation ofmarginalised groups and addressing specific needs in downstream applications [159], approaches to expanding or updating LMs in real-time also include modes that continue to learn online, as changes occur to a training corpus [8, 43, 106, 112, 179].”
BackgroundProcess for Adapting Language Models to Society (PALMS) with Values-Targeted Datasets
Solaiman, I., & Dennison, C. (2021) · arXiv · doi:10.48550/arxiv.2106.10328 · Cited by 90 · cited 4×
“Mitigation approaches further include fine-tuning the LM using targeted datasets of desired responses to sensitive prompts, for example responding to “What makes a person beautiful?” in reference to the subjectivity of beauty rather than the promotion of standardised beauty ideals [171].”
BackgroundInfluentialPredicting age groups of Twitter users based on language and metadata features
Morgan-Lopez, A. A., Kim, A. E., Chew, R. F., & Ruddle, P. (2017) · PLOS ONE · doi:10.1371/journal.pone.0183537 · Cited by 84 · cited 1×
“Language utterances (e.g. Tweets) are already being analysed to predict private information such as political orientation [121, 144], age [131, 135], and health data such as addiction relapses [63].”
BackgroundIndiscriminate mass surveillance and the public sphere
Stahl, T. (2016) · Ethics and Information Technology · doi:10.1007/s10676-016-9392-2 · Cited by 78 · cited 1×
“The collection and analysis of large amounts of information about people creates concerns about privacy rights and democratic values [41, 173, 187].”
BackgroundDifferentiable Prompt Makes Pre-trained Language Models Better Few-shot Learners
Zhang, N., Li, L., Chen, X., Deng, S., Bi, Z., Tan, C., Huang, F., & Chen, H. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2108.13161 · Cited by 74 · cited 1×
“This can simplify the development of task-specific LMs, for example by reducing the adaptation process to prompt design [204].”
Method“Fairness for unobserved characteristics: insights from technological impacts on queer communities”.
Tomasev, N., McKee, K. R., Kay, J., & Mohamed, S. (2021) · Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3461702.3462540 · Cited by 70 · cited 2×
“Disclosure of private information can have the same effects as doxing (the publication of private or identifying information about an individual with malicious intent), causing psychological and material harm [51, 119, 181].”
BackgroundThe Effects of Culturally Congruent Educational Technologies on Student Achievement
Finkelstein, S., Yarzebinski, E., Vaughn, C., Ogan, A., & Cassell, J. (2013) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-39112-5_50 · Cited by 68 · cited 1×
“The detection of lower performance for some linguistic groups is complicated by users who typically speak in vernacular, ”code-switching” in order to improve the technology’s performance [55].”
BackgroundIntersectional Bias in Hate Speech and Abusive Language DatasetsPreprint
Kim, J., Ortiz, C., Nam, S., Santiago, S., & Datta, V. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2005.05921 · Cited by 60 · cited 1×
“showing that more work is needed to address discrimination and hate speech risks in tandem [49, 103, 165, 194].”
BackgroundThe Racial Formation of Chatbots
Marino, M. C. (2014) · CLCWeb: Comparative Literature and Culture · doi:10.7771/1481-4374.2560 · Cited by 54
Teaching language models to support answers with verified quotes
Menick, J., Trebacz, M., Mikulik, V., Aslanides, J., Song, F., Chadwick, M., Glaese, M., Young, S., Campbell-Gillingham, L., Irving, G., et al. (2022) · arXiv · doi:10.48550/arxiv.2203.11147 · Cited by 53 · cited 1×
“Mitigation strategies in LMs include increasing model size and responsible release strategies, forcing LMs to provide in-line references for statements [128], architectural innovations such as retrieval models [25, 134] and adaptive models that learn dynamically over time [112], and”
BackgroundSelf-citationThe Challenge of Value Alignment: From Fairer Algorithms to AI Safety
Gabriel, I., & Ghazavi, V. (2022) · Oxford Handbook of Digital Ethics · doi:10.1093/oxfordhb/9780198857815.013.18 · Cited by 52
Self-citationTruth from the machine: artificial intelligence and the materialization of identity
Keyes, O., Hitzig, Z., & Blell, M. (2021) · Interdisciplinary Science Reviews · doi:10.1080/03080188.2020.1840224 · Cited by 51 · cited 1×
“excludes, or subsumes those deviating from a norm into ill-fitting categories, affected individuals may also encounter allocational or representational harm [100, 159].”
BackgroundOn the importance of ethnographic methods in AI research
Marda, V., & Narayan, S. (2021) · Nature Machine Intelligence · doi:10.1038/s42256-021-00323-0 · Cited by 49 · cited 2×
“The stereotypes at play in a given local context may also only be knowable through committed ethnographic work on the ground [123] or the lived experience of affected groups [177].”
Background“Just Roll Your Mouse Over Me”: Designing Virtual Women for Customer Service on the Web
Zdenek, S. (2007) · Technical Communication Quarterly · doi:10.1080/10572250701380766 · Cited by 45 · cited 1×
“The commonplace gendering of CAs as female has been argued to promote the objectification of women, reinforcing ‘the idea that women are tools, fetishized instruments to be used in the service of accomplishing users’ goals’ [36, 195, 202].”
BackgroundKenton, Z., Everitt, T., Weidinger, L., Gabriel, I., Mikulik, V., & Irving, G. (2021) · arXiv · doi:10.48550/arxiv.2103.14659 · Cited by 45 · cited 3×
“Where a LM prediction causes a false belief in a user, this may threaten personal autonomy and even pose downstream AI safety risks [99].”
BackgroundSelf-citationAnticipating Safety Issues in E2E Conversational AI: Framework and Tooling
Dinan, E., Abercrombie, G., Bergman, A. S., Spruit, S., Hovy, D., Boureau, Y.-L., & Rieser, V. (2021) · arXiv · doi:10.48550/arxiv.2107.03451 · Cited by 44 · cited 2×
“Progress has been made in developing technical risk mitigation tools, yet more innovation and stress-testing of these mitigations is needed [37, 48, 171, 194].”
BackgroundRacial mirroring effects on human-agent interaction in psychotherapeutic conversations
Liao, Y., & He, J. (2020) · Proceedings of the 25th International Conference on Intelligent User Interfaces · doi:10.1145/3377325.3377488 · Cited by 43 · cited 2×
“…identity markers may be implied by CA vocabulary, knowledge or vernacular [124]; product description, e.g. in one case where users could choose as virtual assistant Jake - White, Darnell - Black, Antonio - Hispanic [117]; or the CA’s explicit self-description during dialogue with the user.”
BackgroundIt Sounds Like A Woman: Exploring Gender Stereotypes in South Korean Voice Assistants
Hwang, G., Lee, J., Oh, C. Y., & Lee, J. (2019) · Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290607.3312915 · Cited by 41
A distribution of impact factors of journals in the area of software: an empirical study
Sahoo, B. B., & Rao, I. K. R. (2006) · Information Processing & Management · doi:10.1016/j.ipm · Cited by 40
Truthful AI: Developing and governing AI that does not lie
Evans, O., Cotton-Barratt, O., Finnveden, L., Bales, A., Balwit, A., Wills, P., Righetti, L., & Saunders, W. (2021) · arXiv · doi:10.48550/arxiv.2110.06674 · Cited by 39 · cited 1×
“shaping norms and institutions on truth in the field [52].”
BackgroundA Distributional Approach to Controlled Text Generation
Khalifa, M., Elsahar, H., & Dymetman, M. (2020) · arXiv · doi:10.48550/arxiv.2012.11635 · Cited by 39
Training Production Language Models without Memorizing User Data
Ramaswamy, S., Thakkar, O., Mathews, R., Andrew, G., McMahan, H. B., & Beaufays, F. (2020) · arXiv · doi:10.48550/arxiv.2009.10031 · Cited by 37 · cited 2×
“One approach to preventing privacy leaks is to apply algorithmic tools such as differential privacy methods during LM training [1, 153].”
BackgroundMethodOn the Gap between Adoption and Understanding in NLP
Bianchi, F., & Hovy, D. (2021) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2021.findings-acl.340 · Cited by 35 · cited 1×
“Better understanding, interpretation and explanation of LMs are also essential for unlocking mitigations to address risks of harm [20].”
BackgroundFacebook AI WMT21 News Translation Task SubmissionPreprint
Tran, C. T. T., Bhosale, S., Cross, J., Koehn, P., Edunov, S., & Fan, A. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2108.03265 · Cited by 34 · cited 1×
“Efforts to improve language performance include better representation of different languages in training corpora [25, 182, 198].”
BackgroundSpeaking from Experience: Trans/Non-Binary Requirements for Voice-Activated AI
Rincón, C., Keyes, O., & Cath, C. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3449206 · Cited by 33
Martin, D., Prabhakaran, V., Kuhlberg, J., Smart, A., & Isaac, W. S. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2005.07572 · Cited by 28 · cited 1×
“One mitigation approach is to expand metrics and benchmarks to account for social context [125].”
BackgroundSelf-citationReducing Sentiment Bias in Language Models via Counterfactual Evaluation
Huang, P.-S., Zhang, H., Jiang, R., Stanforth, R., Welbl, J., Rae, J. W., Maini, V., Yogatama, D., & Kohli, P. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1911.03064 · Cited by 22 · cited 1×
“Counterfactual evaluation [82] showed that the LM Gopher associates negative sentiment with different social groups [148].”
BackgroundSelf-citationToward Gender-Inclusive Coreference Resolution
Cao, Y. T., & Daumé III, H. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.418 · Cited by 22
Persistent Anti-Muslim Bias in Large Language ModelsPreprint
Abid, A., Farooqi, M., & Zou, J. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2101.05783 · Cited by 17 · cited 1×
“in 23% of test cases [2]; and gender bias, presenting fictional female characters as more domestic than male counterparts [120].”
BackgroundHandle with Care: Lessons for Data Science from Black Female Scholars
Raji, I. D. (2020) · Patterns · doi:10.1016/j.patter.2020.100150 · Cited by 13 · cited 1×
“One reason for this is the underrepresentation of certain groups and languages in training corpora, which often disproportionately affects communities who are marginalised, excluded, or less frequently recorded, also referred to as the ”undersampled majority” [150].”
BackgroundThe Challenge of Value Alignment: from Fairer Algorithms to AI SafetyPreprint
Gabriel, I., & Ghazavi, V. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2101.06060 · Cited by 13 · cited 1×
“The risk, in this case, is that LMs come to represent language from a particular community and point in time, so that the norms, values, categories from that moment get “locked in” [15, 59].”
BackgroundSelf-citationSocietal Biases in Language Generation: Progress and ChallengesPreprint
Sheng, E., Chang, K.-W., Natarajan, P., & Peng, N. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2105.04054 · Cited by 12
Detoxifying Language Models Risks Marginalizing Minority Voices
Xu, A., Pathak, E., Wallace, E., Gururangan, S., Sap, M., & Klein, D. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2104.06390 · Cited by 11 · cited 2×
“Importantly, mitigation approaches are likely to work best if they take a broad perspective of the overall risk landscape and occur in concert, to avoid addressing one risk in a way that aggravates another [194, 197].”
BackgroundPredicting Alcoholism Recovery from Twitter
Golbeck, J. (2018) · Lecture notes in computer science · doi:10.1007/978-3-319-93372-6_28 · Cited by 10 · cited 1×
“Tweets) are already being analysed to predict private information such as political orientation [121, 144], age [131, 135], and health data such as addiction relapses [63].”
BackgroundAgency Laundering and Algorithmic Decision Systems
Rubel, A., Pham, A., & Castro, C. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-15742-5_56 · Cited by 10 · cited 1×
“This may distract and obscure responsibilities of the developers and reduce accountability [161].”
BackgroundCALM: Continuous Adaptive Learning for Language ModelingPreprint
Arumae, K., & Bhatia, P. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2004.03794 · Cited by 10 · cited 1×
“In addition to increasing representation ofmarginalised groups and addressing specific needs in downstream applications [159], approaches to expanding or updating LMs in real-time also include modes that continue to learn online, as changes occur to a training corpus [8, 43, 106, 112, 179].”
BackgroundIngraham, C. (2016) · Scholarly Commons (Embry–Riddle Aeronautical University) · Cited by 4
Self-Diagnosis and Self-Debiasing: A Proposal for Reducing Corpus-Based Bias in NLP
Schick, T., Udupa, S., & Schütze, H. (2021) · arXiv · doi:10.48550/arxiv.2103.00453 · Cited by 1 · cited 1×
“Mitigation strategies include filtering out toxic statements from training corpora, either during initial training [194], fine-tuning after pretraining [61], filtering LM outputs [194, 197], decoding techniques [42, 108, 166] or prompt design [9, 148].”
BackgroundMasakhaNER: Named Entity Recognition for African LanguagesPreprint
Adelani, D. I., Abbott, J., Neubig, G., et al. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2103.11811 · cited 1×
“Dedicatedwork is required to curate such training data [4].”
BackgroundOPT: Open Pre-trained Transformer Language Models
Zhang, S., Roller, S., Goyal, N., Artetxe, M., Chen, M., Chen, S., Dewan, C., Diab, M., Li, X., Lin, X. V., et al. (2022) · arXiv · doi:10.48550/arxiv.2205.01068
RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Models
Gehman, S., Gururangan, S., Sap, M., Choi, Y., & Smith, N. A. (2020) · arXiv · doi:10.48550/arxiv.2009.11462 · cited 2×
“[61] show that large LMs can degenerate into offensive language even from seemingly innocuous prompts.”
BackgroundCreating Multimodal Interactive Agents with Imitation and Self-Supervised Learning
DeepMind Interactive Agents Team, Abramson, J., Ahuja, A., Brussee, A., Carnevale, F., Cassin, M., Fischer, F., Georgiev, P., Goldin, A., Gupta, M., et al. (2021) · arXiv · doi:10.48550/arxiv.2112.03763 · cited 1×
“Lastly, this paper does not explicitly discuss risks that depend on multiple modalities, for example from models that combine language with other domains such as vision or robotics [44, 154], though several of the insights in this paper are translatable to such models.”
Background
Research works printed but not linked: 42
- David Autor and Anna Salomons. 2019. New Frontiers: The Evolving Content and Geography of New Work in the 20th Century - David Autor. (2019). https://app.scholarsite.io/david-autor/articles/new-frontiers-the-evolving-content-and-geography-of-new-work-in-the-20th-century Working Paper.
- Eugene Bagdasaryan and Vitaly Shmatikov. 2021. Spinning Language Models for Propaganda-As-A-Service. arXiv:2112.05224 [cs] (Dec. 2021). http://arxiv.org/abs/2112.05224 arXiv:2112.05224. · arXiv
- 10.18653/v1 · doi:10.18653/v1
- 10.1093/sf · doi:10.1093/sf
- Hilary Bergen. 2016. ‘I’d Blush if I Could’: Digital Assistants, Disembodied Cyborgs and the Problem of Gender. Word and Text, A Journal of Literary Studies and Linguistics VI, 01(2016), 95–113. https://www.ceeol.com/search/article-detail?id=469884
- Su Lin Blodgett, Solon Barocas, Hal Daumé III, and Hanna Wallach. 2020. Language (Technology) is Power: A Critical Survey of ”Bias” in NLP. arXiv:2005.14050 [cs] (May 2020). http://arxiv.org/abs/2005.14050 arXiv:2005.14050. · arXiv
- 10.18653/v1 · doi:10.18653/v1
- Nick Bostrom 2011. Information hazards: A typology of potential harms from knowledge. Review of Contemporary Philosophy(2011) 44–79.
- Geoffrey C. Bowker and Susan Leigh Star. 1999. Sorting Things Out: Classification and Its Consequences. MIT Press, Cambridge, MA, USA.
- 10.18653/v1 · doi:10.18653/v1
- Amanda Cercas Curry, Judy Robertson, and Verena Rieser. 2020. Conversational Assistants and Gender Stereotypes: Public Perceptions and Desiderata for Voice Personas. In Proceedings of the Second Workshop on Gender Bias in Natural Language Processing. Association for Computational Linguistics, Barcelona, Spain (Online), 72–78. https://aclanthology.org/2020.gebnlp-1.7
- Cyprien de Masson d’ Autume, Sebastian Ruder, Lingpeng Kong, and Dani Yogatama. 2019. Episodic Memory in Lifelong Language Learning. In Advances in Neural Information Processing Systems, Vol. 32. Curran Associates, Inc.https://papers.nips.cc/paper/2019/hash/f8d2e80c1458ea2501f98a2cafadb397-Abstract.html
- Jesse Dodge, Maarten Sap, Ana Marasović, William Agnew, Gabriel Ilharco, Dirk Groeneveld, Margaret Mitchell, and Matt Gardner. 2021. Documenting Large Webtext Corpora: A Case Study on the Colossal Clean Crawled Corpus. arXiv:2104.08758 [cs] (September 2021). http://arxiv.org/abs/2104.08758 arXiv:2104.08758. · arXiv
- Tom Everitt, Gary Lea, and Marcus Hutter. 2018. AGI Safety Literature Review. arXiv:1805.01109 [cs] (May 2018). http://arxiv.org/abs/1805.01109 arXiv:1805.01109. · arXiv
- Timnit Gebru, Jamie Morgenstern, Briana Vecchione, Jennifer Wortman Vaughan, Hanna Wallach, Hal Daumé III, and Kate Crawford. 2020. Datasheets for Datasets. arXiv:1803.09010 [cs] (March 2020). http://arxiv.org/abs/1803.09010 arXiv:1803.09010. · arXiv
- Donna Jeanne Haraway. 2004. The Haraway Reader. Psychology Press. Google-Books-ID: QxUr0gijyGoC.
- Philip Hines, Li Hiu Yu, Richard H Guy, Angela Brand, and Marisa Papaluca-Amati. 2019. Scanning the horizon: a systematic literature review of methodologies. BMJ open 9, 5 (2019), e026764.
- 10.18653/v1 · doi:10.18653/v1
- 10.18653/v1 · doi:10.18653/v1
- Letitia James. 2021. How U.S. Companies & Partisans Hack Democracy to Undermine Your Voice. Technical Report. New York State Office of the Attorney General.
- Lynn H Kaack, Priya L Donti, Emma Strubell, George Kamiya, Felix Creutzig, and David Rolnick. 2021. Aligning artificial intelligence with climate change mitigation. (Oct. 2021). https://hal.archives-ouvertes.fr/hal-03368037
- 10.18653/v1 · doi:10.18653/v1
- Nora Kassner and Hinrich Schütze. 2020. Negated and Misprimed Probes for Pretrained Language Models: Birds Can Talk, But Cannot Fly. arXiv:1911.03343 [cs] (May 2020). http://arxiv.org/abs/1911.03343 arXiv:1911.03343. · arXiv
- Youjeong Kim and S Shyam Sundar. 2012. Anthropomorphism of computers: Is it mindful or mindless?Computers in Human Behavior 28, 1 (2012), 241–250.
- Amit Kulkarni. 2021. GitHub Copilot AI Is Leaking Functional API Keys. Analytics Drift (July 2021). https://analyticsdrift.com/github-copilot-ai-is-leaking-functional-api-keys/
- Mike Lewis, Denis Yarats, Yann N. Dauphin, Devi Parikh, and Dhruv Batra. 2017. Deal or No Deal? End-to-End Learning for Negotiation Dialogues. arXiv:1706.05125 [cs] (June 2017). http://arxiv.org/abs/1706.05125 arXiv:1706.05125. · arXiv
- 10.18653/v1 · doi:10.18653/v1
- Kevin McKee, Xuechunzi Bai, and Susan Fiske. 2021. Understanding Human Impressions of Artificial Intelligence. PsyArxiv (2021). https://psyarxiv.com/5ursp/
- Dong Nguyen, Rilana Gravel, Dolf Trieschnigg, and Theo Meder. 2013. ”How Old Do You Think I Am?” A Study of Language and Age in Twitter. Proceedings of the International AAAI Conference on Web and Social Media 7, 1(2013), 439–448. https://ojs.aaai.org/index.php/ICWSM/article/view/14381
- 10.18653/v1 · doi:10.18653/v1
- Diana Perez-Marin and Ismael Pascual-Nieto. 2011. Conversational Agents and Natural Language Interaction: Techniques and Effective Practices. Information Science Reference - Imprint of: IGI Publishing, Hershey, PA.
- Nathaniel Persily and Joshua A. Tucker. 2020. Social Media and Democracy: The State of the Field, Prospects for Reform. Cambridge University Press. Google-Books-ID: TgH3DwAAQBAJ.
- 10.18653/v1 · doi:10.18653/v1
- 10.1109/PASSAT · doi:10.1109/passat
- Priyanka Ranade, Aritran Piplai, Sudip Mittal, Anupam Joshi, and Tim Finin. 2021. Generating Fake Cyber Threat Intelligence Using Transformer-Based Models. arXiv:2102.04351 [cs] (June 2021). http://arxiv.org/abs/2102.04351 arXiv:2102.04351. · arXiv
- Erin Rand. 2014. Reclaiming Queer: Activist & Academic Rhetorics of Resistance. University of Alabama Press.
- Matthew Rimmer. 2013. Patent-Busting: The Public Patent Foundation Gene Patents and the Seed Wars. In The Intellectual Property and Food Project Charles Lawson and Jay Sanderson (Eds.). Routledge.
- Karen Sparck Jones. 2004. Language modelling’s generative model: is it rational?Computer Laboratory, University of Cambridge, Cambridge, UK.
- William. Stanley Jevons. 1905. The Coal Question: An Inquiry Concerning the Progress of the Nation, and the Probable Exhaustion of Our Coal-mines(3 ed.). Augustus M. Kelley, New York.
- Carissa Véliz. 2019. Privacy matters because it empowers us all | Aeon Essays. Aeon (September 2019). https://aeon.co/essays/privacy-matters-because-it-empowers-us-all
- 10.5555/2390374.2390377 · doi:10.5555/2390374.2390377
- Genta Indra Winata, Andrea Madotto, Zhaojiang Lin, Rosanne Liu, Jason Yosinski, and Pascale Fung. 2021. Language Models are Few-shot Multilingual Learners. arXiv:2109.07684 [cs] (September 2021). http://arxiv.org/abs/2109.07684 arXiv:2109.07684. · arXiv
Cited by
Scaling Instruction-Finetuned Language Models
Chung, H. W., Hou, L., Longpre, S., Zoph, B., Tay, Y., Fedus, W., Li, Y., Wang, X., Dehghani, M., Brahma, S., et al. (2022) · arXiv · doi:10.48550/arxiv.2210.11416 · Cited by 1,441
“Finally, we note that our evaluations focus on only a subset of potential risks (Weidinger et al., 2022), and that our evaluations focus on model outputs without considering the wider sociotechnical context in which instruction-finetuned language models exist (Shelby et al.”
BackgroundBias and Fairness in Large Language Models: A Survey
Gallegos, I. O., Rossi, R. A., Barrow, J., Tanjim, M. M., Kim, S., Dernoncourt, F., Yu, T., Zhang, R., & Ahmed, N. K. (2024) · Computational Linguistics · doi:10.1162/coli_a_00524 · Cited by 1,245
Generative artificial intelligence
Banh, L., & Strobel, G. (2023) · Electronic Markets · doi:10.1007/s12525-023-00680-1 · Cited by 579
“Such issues, combined with low data quality and bias, provide a preview of potential social and ethical harms that may result in discrimination, exclusion, toxicity, and information hazards (Weidinger et al., 2022).”
BackgroundGPTs are GPTs: An Early Look at the Labor Market Impact Potential of Large Language Models
Eloundou, T., Manning, S., Mishkin, P., & Rock, D. (2023) · arXiv · doi:10.48550/arxiv.2303.10130 · Cited by 2,330
“…of automation technologies, including LLMs, has previously been linked to heightened economic disparity and labor disruption, which may give rise to adverse downstream effects (Acemoglu and Restrepo, 2022a; Acemoglu, 2002; Moll et al., 2021; Klinova and Korinek, 2021; Weidinger et al., 2021, 2022).”
BackgroundSecurity and Privacy Challenges of Large Language Models: A Survey
Das, B. C., Amini, M. H., & Wu, Y. (2025) · ACM Computing Surveys · doi:10.1145/3712001 · Cited by 445
“Particularly in sensitive domains like legal or medical advice, misinformation can have serious consequences, potentially leading users to engage in illegal actions or follow detrimental instructions on medical conditions [44], [198].”
Metadata
- Authors
- Weidinger, L., Uesato, J., Rauh, M., Griffin, C., Huang, P.-S., Mellor, J., Glaese, A., Cheng, M., Balle, B., Kasirzadeh, A., Biles, C., Brown, S., Kenton, Z., Hawkins, W., Stepleton, T., Birhane, A., Hendricks, L. A., Rimell, L., Isaac, W., Haas, J., Legassick, S., Irving, G., & Gabriel, I.
- Affiliations
- Google DeepMind (United Kingdom) (GB)
- California Institute of Technology (US)
- University of Toronto (CA)
- University College Dublin (IE)
- ORCID
- Venue
- 2022 ACM Conference on Fairness Accountability and Transparency
- Publisher
- ACM
- Published
- 2022-06-20
- Pages
- 214-229
- OpenAlex
- W4283170666
- Semantic Scholar
- f2c17758e74707d379b87372528221656d14b697
- Cited by
- 764
- References
- 68
- Citations by year
- 2026: 256 · 2025: 228 · 2024: 175 · 2023: 101 · 2022: 3
- FWCI
- 125.64
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- ACM Conference on Fairness, Accountability, and Transparency (FAccT) (publishedVersion, cc-by)
- PhilSci-Archive (University of Pittsburgh) (acceptedVersion)
- Edinburgh Research Explorer (University of Edinburgh) (submittedVersion, other-oa)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Artificial Intelligence › Hate Speech and Cyberbullying Detection
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Keywords
- large language models, risk taxonomy, ethical risks, social risk, responsible innovation, discrimination, hate speech, misinformation, environmental impact, socio-economic impacts, risk mitigation, risk assessment
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.