Persistent Anti-Muslim Bias in Large Language Models
Open accessThe DOI matches its Crossref record; 13 of the 13 references it cites were found in a registry or an index.
Abstract
It has been observed that large-scale language models capture undesirable societal biases, e.g. relating to race and gender; yet religious bias has been relatively unexplored. We demonstrate that GPT-3, a state-of-the-art contextual language model, captures persistent Muslim-violence bias. We probe GPT-3 in various ways, including prompt completion, analogical reasoning, and story generation, to understand this anti-Muslim bias, demonstrating that it appears consistently and creatively in different uses of the model and that it is severe even compared to biases about other religious groups. For instance, Muslim is analogized to terrorist in 23% of test cases, while Jewish is mapped to its most common stereotype, money, in 5% of test cases. We quantify the positive distraction needed to overcome this bias with adversarial text prompts, and find that use of the most positive 6 adjectives reduces violent completions for Muslims from 66% to 20%, but which is still higher than for other religious groups.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Pre-trained on massive amounts of text on the Internet, LMs are known to produce linguistically fawed, factually incorrect, or even ethically problematic text at times [1,24].”
From CoAuthor: Designing a Human-AI Collaborative Writing Dataset for Exploring Language Model Capabilities (2022) · cites this paper as [1]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“LLMs and other AI tools may have a number of biases that could arise from various factors, such as unbalanced training datasets74–77.”
From On opportunities and challenges of large multimodal foundation models in education (2025) · cites this paper as [75]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“it is also a liability; all AI systems might inherit the same problematic biases of a few foundation models [Bolukbasi et al. 2016; Caliskan et al. 2017; Abid et al. 2021, inter alia]”
From On the Opportunities and Risks of Foundation Models (2021) · cites this paper as Abid et al. 2021
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Previous work has shown that language models tend to exhibit human-like biases with respect to specific protected groups such as gender, ethnicity or religious orientation3–6.”
From Generative language models exhibit social identity biases (2024) · cites this paper as [4]
In the citing paper
“If a medical LLM has been trained on data that contains misinformation or biased content, LLMs deployed for clinical decision-making are at risk of reproducing these problematic contents [120,124–126].”
From Bias in medical AI: Implications for clinical decision-making (2024) · cites this paper as [126]
1 more claim about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404 · cited 1×
“These languagemodels have evolved, from learning individual word vectors with single-layer models [8], to more complex language generation architectures such as recurrent neural networks [5] and most recently transformers [4, 6, 14].”
BackgroundEfficient Estimation of Word Representations in Vector Space
Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1301.3781 · Cited by 26,568 · cited 1×
“These languagemodels have evolved, from learning individual word vectors with single-layer models [8], to more complex language generation architectures such as recurrent neural networks [5] and most recently transformers [4, 6, 14].”
BackgroundLanguage Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372 · cited 4×
“These languagemodels have evolved, from learning individual word vectors with single-layer models [8], to more complex language generation architectures such as recurrent neural networks [5] and most recently transformers [4, 6, 14].”
BackgroundMethodInfluentialSelf-Critical Sequence Training for Image Captioning
Rennie, S. J., Marcheret, E., Mroueh, Y., Ross, J., & Goel, V. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.131 · Cited by 2,112 · cited 1×
“These descriptions are generated by passing images through a pretrained image captioning model [11], and fine-tuning GPT-3 to transform the short captions into long descriptions, as shown in Fig.”
MethodReformer: The Efficient Transformer
Kitaev, N., Kaiser, Ł., & Levskaya, A. (2020) · arXiv · doi:10.48550/arxiv.2001.04451 · Cited by 1,435 · cited 1×
“These languagemodels have evolved, from learning individual word vectors with single-layer models [8], to more complex language generation architectures such as recurrent neural networks [5] and most recently transformers [4, 6, 14].”
BackgroundSemi-supervised Sequence Learning
Dai, A. M., & Le, Q. V. (2015) · arXiv · doi:10.48550/arxiv.1511.01432 · Cited by 994
StereoSet: Measuring stereotypical bias in pretrained language models
Nadeem, M., Bethke, A., & Reddy, S. (2020) · arXiv · doi:10.48550/arxiv.2004.09456 · Cited by 643 · cited 2×
“As a result, even though the various language models have different architectures, since they are trained on similar corpora of text, they often learn similar biases [9].”
BackgroundUniversal Adversarial Triggers for Attacking and Analyzing NLP
Wallace, E., Feng, S., Kandpal, N., Gardner, M., & Singh, S. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1221 · Cited by 642
Identifying and Reducing Gender Bias in Word-Level Language Models
Bordia, S., & Bowman, S. R. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Student Research Workshop · doi:10.18653/v1/n19-3002 · Cited by 219 · cited 1×
“Previous work has explored the biases that are ultimately learned by language models, with the primary focus being racial and gender stereotypes [2, 3, 7, 12].”
BackgroundThe Woman Worked as a Babysitter: On Biases in Language GenerationPreprint
Sheng, E., Chang, K.-W., Natarajan, P., & Peng, N. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.01326 · Cited by 205 · cited 2×
“In [12], the authors made similar observations about language generation models.”
Background
Research works printed but not linked: 6
- 7. T. Bolukbasi, K.-W. Chang, J. Y. Zou, V. Saligrama, A. T. Kalai, Advances in neural information processing systems 29 , 4349 (2016).
- 9. S. Bordia, S. R. Bowman, arXiv preprint arXiv:1904.03035 (2019). · arXiv
- 2. A. M. Dai, Q. V. Le, Advances in neural information processing systems 28 , 3079 (2015).
- 10. K. Lu, P. Mardziel, F. Wu, P. Amancharla, A. Datta, Logic, Language, and Security (Springer, 2020), pp. 189-202.
- 3. A. Vaswani, et al. , Advances in neural information processing systems (2017), pp. 59986008.
- 14. E. Wallace, S. Feng, N. Kandpal, M. Gardner, S. Singh, arXiv preprint arXiv:1908.07125 (2019). · arXiv
Cited by
Items 401–500 of 986
MarIA and BETO are sexist: evaluating gender bias in large language models for Spanish
(2023) · doi:10.1007/s10579-023-09670-3 · Cited by 16
“A clear example is the recent GPT-3 (Abid et al., 2021) model that shows a bias towards the Muslim religion by associating it with violence in a high number of cases.”
Building Trustworthy AI in Healthcare
Amadi, C., & Ojo, A. (2025) · IEEE Access · doi:10.1109/access.2025.3648410 · Cited by 16
“In addition, LLMs have shown discrimination against females, individuals of the black race, and a disproportionately higher association of violence with Muslims [27], [78], [79].”
Comparative Analysis of Large Language Models in Chinese Medical Named Entity Recognition
Zhu, Z., Zhao, Q. Z., Li, J., Ge, Y., Ding, X., Gu, T., Zou, J., Lv, S., Wang, S., & Yang, J. (2024) · Bioengineering · doi:10.3390/bioengineering11100982 · Cited by 16
“Recent studies have uncovered potential biases and even harmful suggestions proposed by LLMs [54], particularly in terms of gender [55] and race [56].”
BackgroundSelf-citationGeorgiou, G. P. (2024) · Journal of Language and Social Psychology · doi:10.1177/0261927x241298337 · Cited by 16
“LLMs demonstrate various biases associated with gender (Gross, 2023), language (Georgiou, 2024), religion (Abid et al., 2021), politics (Rozado, 2023), and nationality (Venkit et al., 2023) among others.”
Fairness Mediator: Neutralize Stereotype Associations to Mitigate Bias in Large Language Models
Xiao, Y., Liu, A., Liang, S., Liu, X., & Tao, D. (2025) · Proceedings of the ACM on software engineering. · doi:10.1145/3728881 · Cited by 16
“For instance, GPT-3 [10] has been shown to frequently associate Muslims with violent contexts [2, 38], and Microsoft’s AI chatbot Tay infamously produced racist and inappropriate content after interacting with users on social media [8].”
Fairness in graph-based semi-supervised learning
Zhang, T., Zhu, T., Han, M., Chen, F., Li, J., Zhou, W., & Yu, P. S. (2022) · Knowledge and Information Systems · doi:10.1007/s10115-022-01738-w · Cited by 16
“observed that large-scale languagemodels capture undesirable racial bias [1] andVigdor et al.”
BackgroundThe Spectre of Generative AI Over Advertising, Marketing, and BrandingPreprint
Routray, B. B. (2024) · doi:10.22541/au.170534566.63147021/v1 · Cited by 16
In conversation with Artificial Intelligence: aligning language models with human values
Kasirzadeh, A., & Gabriel, I. (2022) · arXiv · doi:10.48550/arxiv.2209.00731 · Cited by 128
“Does the same notion of truth apply across a wide range of conversational contexts, or might standards of truthfulness vary according to the subject under consideration and to relevant conversational norms? Equally, there is widespread concern that the output of large-scale language models is biased [1, 15].”
BackgroundBadChain: Backdoor Chain-of-Thought Prompting for Large Language Models
Xiang, Z., Jiang, F., Xiong, Z., Ramasubramanian, B., Poovendran, R., & Li, B. (2024) · arXiv · doi:10.48550/arxiv.2401.12242 · Cited by 15
“…(Wang et al., 2023a), such as toxic content generation (Wang et al., 2022a; Zou et al., 2023; Jones et al., 2023), stereotype bias (Li et al., 2020; Abid et al., 2021), privacy leakage (Carlini et al., 2021; Panda et al., 2023), vulnerability against adversarial queries (Wang et al., 2020; 2021;…”
Fairness in Large Language Models in Three Hours
Doan, T. V., Wang, Z., Hoang, N., & Zhang, W. (2024) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3627673.3679090 · Cited by 16
“Stereotypical association [1, 5, 32] method assesses bias by measuring the disparity in the rates at which different demographic groups are linked to stereotyped terms ( e.g., occupations) in the text generated by the model in response to a given prompt [32].”
MethodThe ethical evaluation of large language models and its optimization
Lyu, Y., & Du, Y. (2025) · AI and Ethics · doi:10.1007/s43681-024-00654-9 · Cited by 15
Chen, W., Yu, C., Wang, Y., Chen, M., Xu, Y., & Shi, Y. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3757587 · Cited by 15
“If the training data over-represent certain demographic groups or perspectives, the LLM might prioritize specific viewpoints [2, 22].”
The Rise of AI Avatars: Legal Personhood, Rights and Liabilities in an Evolving Metaverse
Cheong, B. C. (2024) · Journal of Digital Technologies and Law · doi:10.21202/jdtl.2024.42 · Cited by 16
Could a Conversational AI Identify Offensive Language?
da Silva, D. A., Louro, H. D. B., Gonçalves, G. S., Marques, J., Dias, L. A. V., da Cunha, A. M., & Tasinaffo, P. M. (2021) · Information · doi:10.3390/info12100418 · Cited by 15
Online dispute resolution: can we leave the initial decision to Large Language Models (LLM)?
Ferrer-Benítez, M. (2022) · Metaverse Basic and Applied Research · doi:10.56294/mr202223 · Cited by 15
“LLMs are trained based on large amounts of data that, in many cases, may contain implicit or explicit biases.(10,11,12) These biases can be the result of discriminatory practices, cultural stereotypes, or partialities of developers and industries.”
BackgroundSeeing Seeds Beyond Weeds: Green Teaming Generative AI for Beneficial UsesPreprint
Stapleton, L., Taylor, J., Fox, S., Wu, T., & Zhu, H. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2306.03097 · Cited by 15
Potential and Challenges of Model Editing for Social DebiasingPreprint
Yan, J., Wang, F., Li, Y., & Zhang, Y. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2402.13462 · Cited by 15
“…Preprocessing techniques aim to detect and eliminate bias and unfairness early on, either within the dataset (Zmigrod et al., 2019; Dinan et al., 2020; Abid et al., 2021; Qian et al., 2022a; Ghanbarzadeh et al., 2023) or prompt (Mattern et al., 2022; Fatemi et al., 2023; Yang et al., 2023).”
BackgroundMore RLHF, More Trust? On The Impact of Preference Alignment On Trustworthiness
Li, A. J., Krishna, S., & Lakkaraju, H. (2025) · International Conference on Learning Representations · doi:10.48550/arxiv.2404.18870 · Cited by 15
“The tendency to generate or agree with over-generalized beliefs about a particular group of people, which are typically disrespectful and have negative societal impact, are considered as the stereotypical bias of LLMs [46, 47, 48, 49].”
Predicting Field Experiments with Large Language ModelsPreprint
Chen, Y., Hu, Y., & Lu, Y. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5389144 · Cited by 16
Cultural Bias in Large Language Models: Evaluating AI Agents through Moral QuestionnairesPreprint
Münker, S. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2507.10073 · Cited by 15
Ecosystem Graphs: The Social Footprint of Foundation Models
Bommasani, R., Soylu, D., Liao, T., Creel, K., & Liang, P. (2023) · doi:10.21203/rs.3.rs-2961271/v1 · Cited by 14
Generative AI and Perceptual Harms: Who's Suspected of using LLMs?
Kadoma, K., Metaxa, D., & Naaman, M. (2025) · Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713897 · Cited by 14
“For example, language models can produce stereotypical text or demeaning language which is often directed historically marginalized groups like women [44, 51], and racial and ethnic minorities [1, 57].”
Ethical Considerations in the Implementation and Usage of Large Language Models
Stefan, R., Căruțașu, G., & Mocan, M. L. (2024) · Lecture notes in networks and systems · doi:10.1007/978-3-031-54671-6_10 · Cited by 14
Künstliche Intelligenz im DaF-Unterricht. Mediation mit KI-Anwendungen fördern.
Hartmann, D. (2024) · Revista Lengua y Cultura · doi:10.29057/lc.v5i10.12218 · Cited by 14
Implementation of Generative Artificial Intelligence in Sociological Research
Drach, V. E., & Torkunova, J. V. (2025) · Discourse · doi:10.32603/2412-8562-2025-11-1-52-70 · Cited by 14
Chen, Y., Wang, Y., Chan, A. B., Li, J., & Lc, R. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-92823-9_7 · Cited by 14
Identity-related Speech Suppression in Generative AI Content Moderation
Proebsting, G., Anigboro, O. I., Crawford, C. M., Metaxa, D., & Friedler, S. A. (2025) · ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization (EAAMO) · doi:10.1145/3757887.3763010 · Cited by 14
Investigating Semi-Automatic Assessment of Data Sets Fairness by Means of Fuzzy Logic
Gallese, C., Scantamburlo, T., Manzoni, L., & Nobile, M. S. (2023) · IEEE Conference on Computational Intelligence in Bioinformatics and Computational Biology (CIBCB) · doi:10.1109/cibcb56990.2023.10264913 · Cited by 14
“…showed how data sets convey social bias in modern AI systems – in particular those based on machine learning – for relevant examples consider unfair outcomes in semantic representation [2], face recognition [3], clinical decisions [4], speech recognition [5] and text generation [6] among others.”
Understanding the Interplay of Scale, Data, and Bias in Language Models: A Case Study with BERTPreprint
Ali, M. A. M., Panda, S., Shen, Q., Wick, M., & Kobren, A. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2407.21058 · Cited by 15
Zong, M., & Krishnamachari, B. (2022) · arXiv · doi:10.48550/arxiv.2212.00857 · Cited by 13
Regional bias in monolingual English language models
Lyu, J., Dost, K., Koh, Y. S., & Wicker, J. S. (2024) · Machine Learning · doi:10.1007/s10994-024-06555-6 · Cited by 13
Transformer-Based Quantification of the Echo Chamber Effect in Online Communities
Ghafouri, V., Alatawi, F., Karami, M., Such, J. M., & Suárez‐Tangil, G. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3687006 · Cited by 13
Mapping the evolution of generative AI: insights from bibliometric research
Pathak, S., & Pallasena, R. K. (2025) · Journal of Decision System · doi:10.1080/12460125.2024.2438612 · Cited by 13
Bhagat, K., Vasisht, K., & Pruthi, D. (2025) · Findings of the Association for Computational Linguistics: NAACL · doi:10.18653/v1/2025.findings-naacl.262 · Cited by 13
Inducing state anxiety in LLM agents reproduces human-like biases in consumer decision-making
Ben‐Zion, Z., Elyoseph, Z., Spiller, T. R., & Lazebnik, T. (2026) · npj Artificial Intelligence · doi:10.1038/s44387-026-00122-1 · Cited by 13
“Indeed, it is well established that LLMs inherit trait-like biases from their human training data, reproducing disparities across domains such as gender 17 , age 18 , race 19 , religion 20 , nationality 21 , occupation 22 , disability 23 and sexual orientation 24 .”
Gendered Grammar or Ingrained Bias? Exploring Gender Bias in Icelandic Language Models
Friðriksdóttir, S. R., & Einarsson, H. (2024) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/2qudr7em2fho · Cited by 13
“…as these large language models acquire linguistic proficiency through data-driven learning, they also absorb and replicate the gender bias, racism, religious prejudices, and queerphobia ingrained within the data (Bolukbasi et al., 2016; Abid et al., 2021; Kurita et al., 2019; Venkit et al., 2022).”
BackgroundWhen artificial intelligence makes everything similar: The risks of content homogenization
Xie, Y., & Xie, Y. (2026) · Chinese Journal of Sociology · doi:10.1177/2057150x261419573 · Cited by 13
Exposing Bias in Online Communities through Large-Scale Language ModelsPreprint
Wald, C., & Pfahler, L. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2306.02294 · Cited by 13
“Large language generation models such as GPT-2 [3] and GPT-3 [4] have been shown to exhibit various kinds of bias, including racial, gender, and religious bias[5][6][7].”
BackgroundMethodInfluentialImplicit Personalization in Language Models: A Systematic StudyPreprint
Jin, Z., Heil, N., Liu, J., Dhuliawala, S., Qi, Y., Schölkopf, B., Mihalcea, R. F., & Sachan, M. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2405.14808 · Cited by 13
Exploring Bias in over 100 Text-to-Image Generative ModelsPreprint
Vice, J., Akhtar, N., Hartley, R. I., & Mian, A. S. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2503.08012 · Cited by 13
“While social biases (e.g., those related to age, race, or gender) are particularly consequential in public-facing applications (Abid et al., 2021; Luccioni et al., 2023; Naik & Nushi, 2023; Seshadri et al., 2023), they are manifestations of broader model biases, observed from a specific viewpoint.”
Born With a Silver Spoon? Investigating Socioeconomic Bias in Large Language ModelsPreprint
Singh, S., Keshari, S., Jain, V., & Chadha, A. R. (2024) · arXiv (Cornell University) · doi:10.13140/rg.2.2.23602.00968 · Cited by 13
In Agents We Trust, but Who Do Agents Trust? Latent Source Preferences Steer LLM Generations
Khan, M. A., Amani, M., Das, S., Ghosh, B., Wu, Q., Gummadi, K. P., Gupta, M., & Ravichander, A. (2026) · International Conference on Learning Representations · doi:10.48550/arxiv.2602.15456 · Cited by 14
A Comprehensive Survey on Bias and Fairness in Large Language Models
Haque, F., Xu, D., & Niu, X. (2025) · Lecture notes in computer science · doi:10.1007/978-981-96-8197-6_7 · Cited by 12
Fair and square? Evaluating fairness of LLM-generated synthetic datasets
Voria, G., Scala, B., Todisco, L., Venditto, C., Giordano, G., Catolino, G., & Palomba, F. (2025) · Information and Software Technology · doi:10.1016/j.infsof.2025.107980 · Cited by 13
Asseri, B., Abdelaziz, E., & Al‐Wabil, A. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2506.18199 · Cited by 12
Münker, S., Schwager, N., & Rettinger, A. (2026) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2026.eacl-long.51 · Cited by 12
Heyn, H.-M., Knauss, E., Malleswaran, I., & Dinakaran, S. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-29786-1_14 · Cited by 12
Understanding latent affective bias in large pre-trained neural language models
Kadan, A., Deepak, P., Bhadra, S., Gangan, M. P., & Lajish, V. L. (2024) · Natural Language Processing Journal · doi:10.1016/j.nlp.2024.100062 · Cited by 13
“3(c)), which we believe may probably be due to the Islamophobia manifested through text, are inline with the experimental results in Abid et al. (2021b) that reports language generated by GPT-3 (Brown et al., 2020) in the context of the Muslim religion are more associated with violence.”
Data Redaction from Conditional Generative Models
Kong, Z., & Chaudhuri, K. (2024) · IEEE Conference on Secure and Trustworthy Machine Learning (SaTML) · doi:10.1109/satml59370.2024.00035 · Cited by 13
“However, in cases where the model owners share the model weights with third parties, they do not have control over whether the filters or editing methods will be used.”
BackgroundGenerating Mental Health Transcripts with SAPE (Spanish Adaptive Prompt Engineering)
Lozoya, D. C., Berazaluce, A., Perches, J. A. B., Lua, E. H., Conway, M., & D’Alfonso, S. (2024) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2024.naacl-long.285 · Cited by 12
“LLMs can manifest various types of biases in their outputs, necessitating an examination to prevent the inadvertent propagation of such biases (Hemmatian and Varshney, 2022; Abid et al., 2021; Cabrera Lozoya et al., 2023).”
BackgroundBirkenmaier, L., Wagner, C., & Lechner, C. M. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2307.02863 · Cited by 13
“Given that computational text-based measures are susceptible to errors and biases inherent to the models themselves (Abid et al., 2021; Liang et al., 2021), we argue that researchers should openly acknowledge and embrace the potential limitations and biases of their measurement methods.”
BackgroundJustifiable Artificial Intelligence: Engineering Large Language Models for Legal ApplicationsPreprint
Wehnert, S. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2311.15716 · Cited by 14
“Also, bias and unfairness may be present due to a bias in the training data (e.g., religion bias [25]).”
BackgroundVision-Language Models Generate More Homogeneous Stories for Phenotypically Black IndividualsPreprint
Lee, M. H. J., & Jeon, S. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2412.09668 · Cited by 12
“LLMs, for instance, often generate text aligned with group-based stereotypes (e.g., Abid et al., 2021; Lucy and Bamman, 2021).”
Exposing Product Bias in LLM Investment RecommendationPreprint
Zhi, Y., Zhang, X., Wang, L., Jiang, S., Ma, S., Guan, X., & Shen, C. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2503.08750 · Cited by 13
“Bias in LLMs. Existing research focuses on the social fairness of large language models, emphasizing the potential biases in model outputs related to gender, race, and other factors (Abid et al., 2021; Coopamootoo and Ng, 2023; Ling, 2024; Gallegos et al., 2023; Ren et al., 2024; Tang et al., 2024).”
The tail wagging the dog: Dataset construction biases of social bias benchmarks
Selvam, N., Dev, S., Khashabi, D., Khot, T., & Chang, K.-W. (2023) · Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) · doi:10.18653/v1/2023.acl-short.118 · Cited by 11
Simulating Conversational Search Users with Parameterized Behavior
Sekulić, I., Lu, L., Bedi, N. S., & Crestani, F. (2024) · Annual International ACM SIGIR Conference on Research and Development in Information Retrieval in the Asia Pacific Region (SIGIR-AP) · doi:10.1145/3673791.3698425 · Cited by 11
“Previous work [1, 11, 22] (see also the next section) highlights how the quality of training data impacts LLM performance.”
Can AI-Attributed News Challenge Partisan News Selection? Evidence from a Conjoint Experiment
Zoizner, A., Matthes, J., Corbu, N., de Vreese, C. H., Esser, F., Koç-Michalska, K., Schemer, C., Theocharis, Y., & Zilinsky, J. (2025) · The International Journal of Press/Politics · doi:10.1177/19401612251342679 · Cited by 11
“AI outputs have been shown to reflect gender and racial biases, and often present politically left-leaning orientations (e.g., Abid et al. 2021; Garg et al. 2018; Rozado 2020, 2023; Rutinowski et al. 2024; Santurkar et al. 2023).”
A Dual-Layered Evaluation of Geopolitical and Cultural Bias in LLMs
Kim, S., & Kim, H. J. (2025) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2025.acl-srw.38 · Cited by 11
Qiyuan, D., Bai, X., Chen, K., Wang, Y., Nie, L., & Zhang, M. (2025) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2025.acl-long.1504 · Cited by 11
“Large Language Models (LLMs) have achieved significant advancements in various domains, accompanied by growing safety concerns (Tan and Celis, 2019; Sheng et al., 2019; Sandbrink, 2023; Abid et al., 2021; Li et al., 2025).”
The Alignment Veto: How Safety Training Suppresses Cultural Knowledge in LLMs
Zahraei, P. S., Tur, G., Hakkani-Tür, D., & Asgari, E. (2026) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/2026.emnlp-main.1874 · Cited by 11
Digital prejudices: an analysis of gender, racial and religious biases in generative AI chatbots
Chua, A. Y. K., Chen, M., Kan, M., & Seoh, W. (2025) · Internet Research · doi:10.1108/intr-10-2024-1536 · Cited by 12
Addressing Healthcare-related Racial and LGBTQ+ Biases in Pretrained Language Models
Xie, S., Hassanpour, S., & Vosoughi, S. (2024) · Findings of the Association for Computational Linguistics: NAACL · doi:10.18653/v1/2024.findings-naacl.278 · Cited by 11
“These biases, encompassing racial, gender, and religious prejudices (Davidson et al., 2019; Vig et al., 2020; Abid et al., 2021), become ∗ Co-corresponding Authors. especially concerning in high-stakes domains like healthcare.”
BackgroundImplanting LLM’s Knowledge via Reading Comprehension Tree for Toxicity Detection
Kang, H., & Qian, T. (2024) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2024.findings-acl.56 · Cited by 11
“However, due to the factors like unsafe training data and policy decisions, it has been observed that LLMs contain toxicity like societal biases (Abid et al., 2021; Fer-rara, 2023; Ray, 2023).”
Exploring the Jungle of Bias: Political Bias Attribution in Language Models via Dependency Analysis
Jenny, D. F., Billeter, Y., Schölkopf, B., & Jin, Z. (2024) · Workshop on NLP for Positive Impact (NLP4PI) · doi:10.18653/v1/2024.nlp4pi-1.15 · Cited by 11
BordIRlines: A Dataset for Evaluating Cross-lingual Retrieval Augmented Generation
Li, B., Haider, S., Luo, F., Agashe, A., & Callison-Burch, C. (2024) · doi:10.18653/v1/2024.wikinlp-1.3 · Cited by 11
Political Bias in LLMs: Unaligned Moral Values in Agent-centric Simulations
Münker, S. (2025) · LDV-Forum/Journal for language technology and computational linguistics · doi:10.21248/jlcl.38.2025.289 · Cited by 11
“Current research raises concerns about potential biases in the training data leading to misrepresentation of certain groups or viewpoints (Abid, Farooqi, & Zou, 2021; Hutchinson et al., 2020).”
SOBR: A Corpus for Stylometry, Obfuscation, and Bias on Reddit
Emmery, C., Miotto, M., Kramp, S., & Kleinberg, B. (2024) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/3dt5cojkzjya · Cited by 11
“…concerns regarding their harm (Bender et al., 2021), and subsequent work on analyzing and mitigating bias (Go-nen and Goldberg, 2019; Kumar et al., 2020; Wang et al., 2020) have rapidly increased (Abid et al., 2021; Felkner et al., 2023; Ghosh and Caliskan, 2023; Kolisko and Anderson, 2023).”
BackgroundAn Analysis of Social Biases Present in BERT Variants Across Multiple LanguagesPreprint
Milios, A., & BehnamGhader, P. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2211.14402 · Cited by 11
“For example, bias in the Anglosphere against Muslims is a well studied phenomenon, and it has been shown that models trained on English language texts inherit this bias [9].”
BackgroundLarge-Scale, Longitudinal Study of Large Language Models During the 2024 US Election SeasonPreprint
Cen, S. H., Ilyas, A., Driss, H., Park, C., Hopkins, A., Podimata, C., & Mądry, A. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2509.18446 · Cited by 11
Foundations of Multisensory Artificial Intelligence
Liang, P. P. (2024) · arXiv (Cornell University) · doi:10.1184/r1/26239949 · Cited by 11
Emotional Support with Conversational AI: Talking to Machines About LifePreprint
Huang, Y., Stodolska, M., & Sultana, S. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2603.22618 · Cited by 11
“Scholars have argued that algorithmic systems may implicitly encode particular theological assumptions, cultural norms, or moral framings, thereby influencing users’ beliefs while lacking accountability, lived experience, or doctrinal grounding [2, 18, 88, 123].”
Yen, R., Xie, Y., Sultanum, N., & Zhao, J. (2025) · Designing Interactive Systems Conference · doi:10.1145/3715336.3735752 · Cited by 10
On the fairness, diversity and reliability of text-to-image generative models
Vice, J., Akhtar, N., Sigal, L., Hartley, R., & Mian, A. S. (2026) · Artificial Intelligence Review · doi:10.1007/s10462-025-11424-2 · Cited by 10
“Regardless of intent, these models can exhibit biases and unfair behavior due to training on large, uncurated datasets [1, 9, 12, 15, 27].”
Ersoy, A., Vizcarra, G., Mayeesha, T. T., & Müller, B. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.175 · Cited by 10
Adaptation of AI Explanations to Users' Roles
Delaunay, J., Largouët, C., Galárraga, L. A., & van Berkel, N. (2023) · VBN Forskningsportal (Aalborg Universitet) · Cited by 10
Better Innovation for a Better World
Dahl, D. W., Noble, C. H., Schreier, M., & Toubia, O. (2025) · Journal of Marketing · doi:10.1177/00222429251322774 · Cited by 10
“…or other forms of data) that may contain biases and stereotypes, such as speci fi c needs or products that are more likely to be associated with certain genders (Bolukbasi et al. 2016), or certain negative traits associated with a particular ethnic or religious group (Abid, Farooqi, and Zou 2021).”
More is more: Addition bias in large language models
Santagata, L., & Nobili, C. D. (2025) · Computers in Human Behavior Artificial Humans · doi:10.1016/j.chbah.2025.100129 · Cited by 10
Revisiting model fairness via adversarial examples
Zhang, T., Zhu, T., Li, J., Zhou, W., & Yu, P. S. (2023) · Knowledge-Based Systems · doi:10.1016/j.knosys.2023.110777 · Cited by 10
Evaluating LLMs for Gender Disparities in Notable PersonsPreprint
Rhue, L. A., Goethals, S., & Sundararajan, A. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2403.09148 · Cited by 10
“Names have also been used as the minimal information to create stereotypes [25, 27, 26], based on the association of names with social identity, and LLMs are known to have stereotypical bias [30, 14].”
MethodExploring Bengali Religious Dialect Biases in Large Language Models with Evaluation PerspectivesPreprint
Wasi, A. T., Islam, R., Islam, M. R., Rafi, T. H., & Chae, D. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2407.18376 · Cited by 11
“Secondly, religious dialects bear significant cultural and social weight within communities; thus, any biases or inaccuracies in language model interpretations could perpetuate stereotypes [1] or lead to misunderstandings, hampering inclusive interactions in different demo-graphics [25].”
BackgroundB-score: Detecting biases in large language models using response historyPreprint
Vo, A., Taesiri, M. R., Kim, D., & Nguyen, A. T. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2505.18545 · Cited by 10
“…have documented biases in numerous dimensions, including demographic biases (e.g. gender, race, religion, culture, etc.) (Brown et al., 2020; Abid et al., 2021; Zhao et al., 2023; Kumar et al., 2024; Shin et al., 2024), political biases (Bang et al., 2024; Potter et al., 2024), geographical…”
Large Language Models (LLMs): A systematic study in Administration and Business
Pessanha, G. R. G., Vieira, A. G., & Brandão, W. C. (2024) · RAM. Revista de Administração Mackenzie · doi:10.1590/1678-6971/eramd240059 · Cited by 9
Bentley, S. V., Evans, D., & Naughtin, C. K. (2025) · Scientific Reports · doi:10.1038/s41598-025-14875-3 · Cited by 9
“Evidence of racial bias in generative AI’s outputs has been demonstrated in various different guises, whether portraying Muslim people as violent, or Jewish people as overly concerned with money 41 .”
Вызовы экономики искусственного интеллекта традиционному рынку труда
Лукичёв, П. М., & Чекмарев, О. П. (2023) · Russian Journal of Innovation Economics · doi:10.18334/vinec.13.2.118137 · Cited by 9
What Is a Person? Emerging Interpretations of AI Authorship and Attribution
Moulaison, H. L. (2023) · Proceedings of the Association for Information Science and Technology · doi:10.1002/pra2.788 · Cited by 9
Minds: Big questions for linguistics in the age of AI
Backus, A., Cohen, M., Cohn, N., Faber, M., Krahmer, E., Laparle, S., Maier, E., van Miltenburg, E., Roelofsen, F., Sciubba, E., et al. (2023) · Linguistics in the Netherlands · doi:10.1075/avt.00094.bac · Cited by 9
SoUnD Framework: Analyzing (So)cial Representation in (Un)structured (D)ata
Díaz, M., Dev, S., Reif, E., Denton, E., & Prabhakaran, V. (2024) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1609/aies.v7i1.31643 · Cited by 9
Mitigating Bias in Conversations: A Hate Speech Classifier and Debiaser with PromptsPreprint
Raza, S., Ding, C., & Pandya, D. A. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2307.10213 · Cited by 9
“A related work [1] shows that GPT-3, a state-of-the-art contextual language model, captures persistent Muslim-violence bias, demonstrating that it appears consistently and creatively in different uses of the model.”
BackgroundEvaluation of Large Language Models: STEM education and Gender StereotypesPreprint
Due, S., Das, S., Andersen, M., López, B. P., Nexø, S. A., & Clemmensen, L. K. H. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2406.10133 · Cited by 9
“…11 types of biases in LLMs: Gender [21, 20, 25, 37, 15, 22, 35, 12, 29, 4, 5], age [4], sexual orientation [12, 5], physical appearance, disabilities [14, 4], nationalities, ethnicity/race [29, 4, 5], socioeconomic status [4], religion [2, 29, 5], culture [28], and intersectional bias [12].”
BackgroundSpennemann, D. H. R. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5393987 · Cited by 9
Whose Name Comes Up? Auditing LLM-Based Scholar RecommendationsPreprint
Barolo, D., Valentin, C., Karimi, F., Galárraga, L. A., Méndez, G. G., & Espín-Noboa, L. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2506.00074 · Cited by 10
“While bias in LLMs is well documented (Abid, Farooqi, and Zou 2021; Weidinger et al. 2021; Gallegos et al. 2024; Navigli, Conia, and Ross 2023), our audit offers a detailed account of how it appears in scholar recommendations.”
Attributional Safety Failures in Large Language Models under Code-Mixed Perturbations
Banerjee, S., Chatterjee, P., Kumar, S., Layek, S., Agrawal, P., Hazra, R., & Mukherjee, A. (2026) · Transactions on machine learning research · doi:10.48550/arxiv.2505.14469 · Cited by 9
“…have shown promise (Winata et al., 2021; Singh et al., 2021; Panda et al., 2020), LLMs still show vulnerabilities, especially in identifying toxic, biased, or culturally inappropriate content within such hybrid text (Gehman et al., 2020; Bender et al., 2021; Meade et al., 2023; Abid et al., 2021).”
Simbeck, K., & Mahran, M. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2509.17665 · Cited by 9
“Some studies have shown that models associate “Muslim” with “terrorist” or “Jewish” with “money” [11], and that Islam is more frequently linked to violence than Christianity in both language and text-to-image models [12].”
Lee, B. H., Lim, S., & Chun, S. Y. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2503.12356 · Cited by 9
“In recent years, the risks of large-scale models in generating inappropriate content have been widely examined [1, 4, 23], and these concerns extend to T2I diffusion models like Stable Diffusion (SD) [10, 47, 49].”
Deciphering Stereotypes in Pre-Trained Language Models
Ma, W., Scheible, H., Wang, B., Veeramachaneni, G., Chowdhary, P., Sun, A., Koulogeorge, A., Wang, L., Yang, D., & Vosoughi, S. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2023.emnlp-main.697 · Cited by 8
“For example, Abid et al. (2021) discovered that GPT-3 (Brown et al., 2020) generates stereotypical associations of Muslims with phrases like “shooting at will” and “bombing.””
BackgroundAutomated Bias Assessment in AI-Generated Educational Content Using CEAT Framework
Peng, J., Shen, W., Rao, J., & Lin, J. (2025) · Communications in computer and information science · doi:10.1007/978-3-031-99264-3_44 · Cited by 8
Dangol, A., Wolfe, R., Dewitt, A., Chickadel, B., Kientz, J. A., & Dasgupta, S. (2025) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3762807 · Cited by 8
EU AI Act Underrepresented and Insufficient to Address the Risk and Vulnerabilities of Generative AI
Jonnala, S., Thomas, N., & parida, P. (2025) · International Journal of Business Analytics · doi:10.4018/ijban.388757 · Cited by 8
Chen, H., Zhao, J., Zheng, S., Zhang, X., Duan, H., & Lü, X. (2025) · BMC Medical Research Methodology · doi:10.1186/s12874-025-02674-3 · Cited by 8
“The potential for biased labels due to biased training data [12], as well as the difficulty in understanding the nuances of natural language in different contexts [13], leads to considerable variation in performance across different tasks, datasets, and labels.”
Metadata
- Authors
- Abid, A., Farooqi, M., & Zou, J.
- Affiliations
- Stanford University (US)
- McMaster University (CA)
- Venue
- Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society
- Publisher
- ACM
- Published
- 2021-07-21
- Pages
- 298-306
- ISSN
- 3065-8365 (ISSN-L)
- OpenAlex
- W3184144760
- Cited by
- 476
- References
- 3
- Citations by year
- 2026: 81 · 2025: 124 · 2024: 105 · 2023: 97 · 2022: 36 · 2021: 33
- FWCI
- 61.36
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- Proceedings of the AAAI/ACM Conference on AI Ethics and Society (publishedVersion)
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Artificial Intelligence › Hate Speech and Cyberbullying Detection
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Keywords
- Islamophobia, religious bias, large language models, GPT-3, analogical reasoning, adversarial prompts, story generation, bias mitigation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.