Persistent Anti-Muslim Bias in Large Language Models
Open accessThe DOI matches its Crossref record; 13 of the 13 references it cites were found in a registry or an index.
Abstract
It has been observed that large-scale language models capture undesirable societal biases, e.g. relating to race and gender; yet religious bias has been relatively unexplored. We demonstrate that GPT-3, a state-of-the-art contextual language model, captures persistent Muslim-violence bias. We probe GPT-3 in various ways, including prompt completion, analogical reasoning, and story generation, to understand this anti-Muslim bias, demonstrating that it appears consistently and creatively in different uses of the model and that it is severe even compared to biases about other religious groups. For instance, Muslim is analogized to terrorist in 23% of test cases, while Jewish is mapped to its most common stereotype, money, in 5% of test cases. We quantify the positive distraction needed to overcome this bias with adversarial text prompts, and find that use of the most positive 6 adjectives reduces violent completions for Muslims from 66% to 20%, but which is still higher than for other religious groups.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Pre-trained on massive amounts of text on the Internet, LMs are known to produce linguistically fawed, factually incorrect, or even ethically problematic text at times [1,24].”
From CoAuthor: Designing a Human-AI Collaborative Writing Dataset for Exploring Language Model Capabilities (2022) · cites this paper as [1]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“LLMs and other AI tools may have a number of biases that could arise from various factors, such as unbalanced training datasets74–77.”
From On opportunities and challenges of large multimodal foundation models in education (2025) · cites this paper as [75]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“it is also a liability; all AI systems might inherit the same problematic biases of a few foundation models [Bolukbasi et al. 2016; Caliskan et al. 2017; Abid et al. 2021, inter alia]”
From On the Opportunities and Risks of Foundation Models (2021) · cites this paper as Abid et al. 2021
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Previous work has shown that language models tend to exhibit human-like biases with respect to specific protected groups such as gender, ethnicity or religious orientation3–6.”
From Generative language models exhibit social identity biases (2024) · cites this paper as [4]
In the citing paper
“If a medical LLM has been trained on data that contains misinformation or biased content, LLMs deployed for clinical decision-making are at risk of reproducing these problematic contents [120,124–126].”
From Bias in medical AI: Implications for clinical decision-making (2024) · cites this paper as [126]
1 more claim about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404 · cited 1×
“These languagemodels have evolved, from learning individual word vectors with single-layer models [8], to more complex language generation architectures such as recurrent neural networks [5] and most recently transformers [4, 6, 14].”
BackgroundEfficient Estimation of Word Representations in Vector Space
Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1301.3781 · Cited by 26,568 · cited 1×
“These languagemodels have evolved, from learning individual word vectors with single-layer models [8], to more complex language generation architectures such as recurrent neural networks [5] and most recently transformers [4, 6, 14].”
BackgroundLanguage Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372 · cited 4×
“These languagemodels have evolved, from learning individual word vectors with single-layer models [8], to more complex language generation architectures such as recurrent neural networks [5] and most recently transformers [4, 6, 14].”
BackgroundMethodInfluentialSelf-Critical Sequence Training for Image Captioning
Rennie, S. J., Marcheret, E., Mroueh, Y., Ross, J., & Goel, V. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.131 · Cited by 2,112 · cited 1×
“These descriptions are generated by passing images through a pretrained image captioning model [11], and fine-tuning GPT-3 to transform the short captions into long descriptions, as shown in Fig.”
MethodReformer: The Efficient Transformer
Kitaev, N., Kaiser, Ł., & Levskaya, A. (2020) · arXiv · doi:10.48550/arxiv.2001.04451 · Cited by 1,435 · cited 1×
“These languagemodels have evolved, from learning individual word vectors with single-layer models [8], to more complex language generation architectures such as recurrent neural networks [5] and most recently transformers [4, 6, 14].”
BackgroundSemi-supervised Sequence Learning
Dai, A. M., & Le, Q. V. (2015) · arXiv · doi:10.48550/arxiv.1511.01432 · Cited by 994
StereoSet: Measuring stereotypical bias in pretrained language models
Nadeem, M., Bethke, A., & Reddy, S. (2020) · arXiv · doi:10.48550/arxiv.2004.09456 · Cited by 643 · cited 2×
“As a result, even though the various language models have different architectures, since they are trained on similar corpora of text, they often learn similar biases [9].”
BackgroundUniversal Adversarial Triggers for Attacking and Analyzing NLP
Wallace, E., Feng, S., Kandpal, N., Gardner, M., & Singh, S. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1221 · Cited by 642
Identifying and Reducing Gender Bias in Word-Level Language Models
Bordia, S., & Bowman, S. R. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Student Research Workshop · doi:10.18653/v1/n19-3002 · Cited by 219 · cited 1×
“Previous work has explored the biases that are ultimately learned by language models, with the primary focus being racial and gender stereotypes [2, 3, 7, 12].”
BackgroundThe Woman Worked as a Babysitter: On Biases in Language GenerationPreprint
Sheng, E., Chang, K.-W., Natarajan, P., & Peng, N. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.01326 · Cited by 205 · cited 2×
“In [12], the authors made similar observations about language generation models.”
Background
Research works printed but not linked: 6
- 7. T. Bolukbasi, K.-W. Chang, J. Y. Zou, V. Saligrama, A. T. Kalai, Advances in neural information processing systems 29 , 4349 (2016).
- 9. S. Bordia, S. R. Bowman, arXiv preprint arXiv:1904.03035 (2019). · arXiv
- 2. A. M. Dai, Q. V. Le, Advances in neural information processing systems 28 , 3079 (2015).
- 10. K. Lu, P. Mardziel, F. Wu, P. Amancharla, A. Datta, Logic, Language, and Security (Springer, 2020), pp. 189-202.
- 3. A. Vaswani, et al. , Advances in neural information processing systems (2017), pp. 59986008.
- 14. E. Wallace, S. Feng, N. Kandpal, M. Gardner, S. Singh, arXiv preprint arXiv:1908.07125 (2019). · arXiv
Cited by
Items 301–400 of 986
Dutta, A., Khorramrouz, A., Dutta, S., & KhudaBukhsh, A. R. (2024) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2024/801 · Cited by 28
“Prior research has reported anti-Muslim bias in large language models [ Abid et al. , 2021 ] .”
BackgroundMethodConcept Pinpoint Eraser for Text-to-image Diffusion Models via Residual Attention Gate
Lee, B. H., Lim, S., Lee, S., Kang, D. U., & Chun, S. Y. (2025) · International Conference on Learning Representations · doi:10.48550/arxiv.2506.22806 · Cited by 28
“The risks of generating inappropriate content by T2I diffusion models, like Stable Diffusion (SD), have been extensively explored (Abid et al., 2021; Birhane & Prabhu, 2021; Hutson, 2021; Rombach et al., 2022b; Rombach, 2022).”
ChatGPT as an assistive technology to enhance reading comprehension for individuals with ADHD
Tamdjidi, R., & Pagès Billai, D. (2023) · KTH Publication Database DiVA (KTH Royal Institute of Technology) · Cited by 28
(2023) · doi:10.52202/075280-3059 · Cited by 27
A CIA Triad-Based Taxonomy of Prompt Attacks on Large Language Models
Jones, N., Whaiduzzaman, M., Jan, T., Adel, A., Alazab, A., & Alkreisat, A. (2025) · Future Internet · doi:10.3390/fi17030113 · Cited by 28
“Additionally, attackers have been observed manipulating LLMs to generate fake news articles that align with specific biases, exacerbating societal polarization [38,39].”
Shihadeh, J., Ackerman, M., Troske, A., Lawson, N. M., & Gonzalez, E. (2022) · IEEE Global Humanitarian Technology Conference (GHTC) · doi:10.1109/ghtc55712.2022.9910995 · Cited by 27
“Studies on the GPT-3 model have revealed bias through gendered associations of occupations, sentiment toward race, and co-occurrence of words with respect to different religions ( [6], [16], [19]).”
BackgroundFairness Definitions in Language Models Explained
Yin, Z., Wang, Z., Palikhe, A., & Zhang, W. (2026) · Wiley Interdisciplinary Reviews Data Mining and Knowledge Discovery · doi:10.1002/widm.70063 · Cited by 29
“Additionally, different evaluation metrics can be employed, including demographic representation [22, 99], stereotypical association [3, 94], counterfactual fairness [93, 94], and performance disparities [142, 165].”
InfluentialPosition: Key Claims in LLM Research Have a Long Tail of Footnotes
Rogers, A., & Luccioni, A. S. (2023) · IT University Of Copenhagen (IT University of Copenhagen) · doi:10.48550/arxiv.2308.07120 · Cited by 28
“Access to both models and their training data is especially important for this purpose, since it can help establish links between LLM behavior and their training data (Piktus et al., 2023) and understand their biases (Gururangan et al., 2022; Johnson et al., 2022; Abid et al., 2021).”
BackgroundFoundation Models in Healthcare: Opportunities, Biases and Regulatory Prospects in Europe
Wójcik, M. A. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-12673-4_3 · Cited by 26
KnowledgeVIS: Interpreting Language Models by Comparing Fill-in-the-Blank Prompts
Coscia, A. J., & Endert, A. (2023) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2023.3346713 · Cited by 27
“For natural language processing (NLP) researchers and engineers who increasingly train and deploy LLMs as “black boxes” for generating text, exploring how learned behaviors dur-ingtrainingmanifestindownstreamtaskscanhelpthemimprovemodeldevelopment;e.g.,bysurfacingharmfulstereotypes[6].”
BackgroundFrom Bytes to Biases: Investigating the Cultural Self-Perception of Large Language Models
Messner, W., Greene, T., & Matalone, J. (2025) · Journal of Public Policy & Marketing · doi:10.1177/07439156251319788 · Cited by 26
“Bias is measured using conno-tative evaluations of sentence completion, word association, and gendered pronoun selection tests (Abid, Farooqi, and Zou 2021; Bubeck et al. 2023; Chowdhery, Narang, and Devlin 2023; Magee et al. 2021; Muralidhar 2021; Nadeem, Bethke, and Reddy 2021; Shihadeh et al.…”
TRUSTLLM:Trustworthiness in Large Language Models
Huang, Y., & Vanschoren, J. (2024) · TU/e Research Portal · Cited by 27
Neuro-symbolic Commonsense Social ReasoningPreprint
Chanin, D., & Anthony Hunter, A. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2303.08264 · Cited by 26
“This is especially problematic when dealing with controversial social norms, where current deep learning models have been shown to learn a range of problematic racial and gender biases (Abid, Farooqi, and Zou 2021; Bolukbasi et al. 2016; Lu et al. 2020).”
BackgroundExploring the Relevance of Data Privacy-Enhancing Technologies for AI Governance Use CasesPreprint
Bluemke, E., Collins, T., Garfinkel, B., & Trask, A. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2303.08956 · Cited by 27
“Additionally, the ‘structured access’ paradigm advocated by Shevlane proposes options for enabling API access to models without full release [3], [44], which may fulfil the information flow needs of some types of external scrutiny [45], [46].”
MethodReason first, then respond: Modular Generation for Knowledge-infused Dialogue
Adolphs, L., Shuster, K., Urbanek, J., Szlam, A., & Weston, J. (2022) · Findings of the Association for Computational Linguistics: EMNLP 2022 · doi:10.18653/v1/2022.findings-emnlp.527 · Cited by 25
Towards Automatic Boundary Detection for Human-AI Collaborative Hybrid Essay in Education
Zeng, Z., Sha, L., Li, Y., Yang, K., Gašević, D., & Chen, G. (2023) · arXiv · doi:10.48550/arxiv.2307.12267 · Cited by 25
“…correct but counterfactual texts that might affect public opinion, e.g., fake news (Zellers et al. 2019), fake app reviews (Martens and Maalej 2019), fake social media text (Fagni et al. 2021), and other harmful text (Weidinger et al. 2021; Abid, Farooqi, and Zou 2021; Gehman et al. 2020).”
BackgroundBias and stereotyping: Human and artificial intelligence (AI)
Kang, O., & Hirschi, K. (2025) · Annual Review of Applied Linguistics · doi:10.1017/s026719052500008x · Cited by 25
Do Large Language Models Favor Recent Content? A Study on Recency Bias in LLM-Based Reranking
Fang, H., Tao, S., Chen, N., Chang, K.-X., & Sakai, T. (2025) · Annual International ACM SIGIR Conference on Research and Development in Information Retrieval in the Asia Pacific Region (SIGIR-AP) · doi:10.1145/3767695.3769493 · Cited by 25
Richer Countries and Richer Representations
Zhou, K., Ethayarajh, K., & Jurafsky, D. (2022) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2022.findings-acl.164 · Cited by 25
FPGNN: Fair path graph neural network for mitigating discrimination
Zhang, G., Cheng, D., & Zhang, S. (2023) · World Wide Web · doi:10.1007/s11280-023-01178-8 · Cited by 25
Exploring Social Biases of Large Language Models in a College Artificial Intelligence Course
Kolisko, S., & Anderson, C. J. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i13.26879 · Cited by 25
“Most of these focus on gender stereotypes.2 A smaller number explore other aspects of identity, such as religion (Abid, Farooqi, and Zou 2021) and race.3 We present the final project to students through the lens of Underwood (2021)’s proposal that LLMs act as models of culture: they distill…”
Manifestations of xenophobia in AI systems
Tomašev, N., Maynard, J. L., & Gabriel, I. (2024) · AI & Society · doi:10.1007/s00146-024-01893-4 · Cited by 25
“Large language models may exhibit varying degrees of positive or negative sentiment across national (Rae et al. 2021; Venkit et al. 2023), societal (Khandelwal et al. 2023) and religious groups (Abid et al. 2021b; Abid et al. 2021), inheriting features of historical and contemporary discourse.”
Blind Spots and Biases: Exploring the Role of Annotator Cognitive Biases in NLP
Gautam, S., & Srinath, M. (2024) · Workshop on Bridging Human-Computer Interaction and Natural Language Processing (HCI+NLP) · doi:10.18653/v1/2024.hcinlp-1.8 · Cited by 26
C3AI: Crafting and Evaluating Constitutions for Constitutional AI
Kyrychenko, Y., Zhou, K., Bogucka, E. P., & Quercia, D. (2025) · International World Wide Web Conference (WWW) · doi:10.1145/3696410.3714705 · Cited by 25
“Despite the rapid assimilation of large language models (LLMs) into the mainstream [49, 66], recent research has shown that LLMs can exhibit harmful behaviors [29] and social, racial, religious, and gender biases [1, 11, 37].”
A Survey on the Impact of Pre-Trained Language Models in Sentiment Classification Task
Gautam, H., Gaur, A., & Yadav, D. K. (2025) · International Journal of Data Science and Analytics · doi:10.1007/s41060-025-00805-z · Cited by 26
Compositional Capabilities of Autoregressive Transformers: A Study on Synthetic, Interpretable TasksPreprint
Ramesh, R., Lubana, E. S., Khona, M., Dick, R. P., & Tanaka, H. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2311.12997 · Cited by 26
“…biased, or toxic outputs (Weidinger et al., 2021; McGuffie & Newhouse, 2020; Garrido-Mu˜noz et al., 2021; Lin et al., 2021; Jiang et al., 2021; Abid et al., 2021; Par-2 rish et al., 2021; Xu et al., 2021; Huang et al., 2019; Sheng et al., 2019; Gehman et al., 2020; Xu et al., 2020; Tamkin et…”
Tackling Bias in Pre-trained Language Models: Current Trends and Under-represented SocietiesPreprint
Yogarajan, V., Dobbie, G., Keegan, T. T., & Neuwirth, R. J. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2312.01509 · Cited by 25
“…modifying prompt language to instruct the model to avoid using stereotypes [122], prepending a positive adjective or short phrases to the prompts [1, 197], use of adversarial triggers [178, 197], controlling tokens in prompts [49], and iterative search of input prompts to select prompts that…”
BackgroundMethodInfluentialPetzel, Z. W., & Sowerby, L. (2025) · Computers in Human Behavior · doi:10.1016/j.chb.2025.108563 · Cited by 24
Hu, W., & Shao, Z. (2025) · Computers in Human Behavior Reports · doi:10.1016/j.chbr.2025.100735 · Cited by 24
Pretraining Language Models with Human Preferences
Korbak, T., Shi, K., Chen, A., Bhalerao, R., Buckley, C. L., Phang, J., Bowman, S. R., & Perez, E. (2023) · arXiv · doi:10.48550/arxiv.2302.08582 · Cited by 23
“Toxicity LMs can generate highly harmful language, including insults, profanities and threats (Sap et al., 2019; Gehman et al., 2020; Abid et al., 2021).”
BackgroundA Hazard Analysis Framework for Code Synthesis Large Language Models
Khlaaf, H., Mishkin, P., Achiam, J., Krueger, G., & Brundage, M. (2022) · arXiv · doi:10.48550/arxiv.2207.14157 · Cited by 23
“• Potential inline generation features being utilized in an open-ended manner permitting general, non-code usage of language model capabilities, mirroring what has been found in the case of other language models trained on Internet data [2, 5, 7, 10].”
BackgroundBlind Faith? User Preference and Expert Assessment of AI-Generated Religious Content
Alam, S., Abdulhai, M., & Salehi, N. (2025) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3715275.3732162 · Cited by 25
“Studies have shown that LLMs like ChatGPT and Gemini exhibit anti-Muslim biases and misquote Quranic verses, issues compounded by their eloquent delivery, which can make errors difficult to detect [2, 16, 24, 38].”
Data Redaction from Pre-trained GANs
Kong, Z., & Chaudhuri, K. (2023) · IEEE Conference on Secure and Trustworthy Machine Learning (SaTML) · doi:10.1109/satml54575.2023.00048 · Cited by 23
“…[Kaneko and Harada, 2021] or checkerboard artifacts [Odena et al., 2016, Zhang et al., 2019, Wang et al., 2020, Schwarz et al., 2021], speech models produce unnatural sound [Donahue et al., 2018, Thiem et al., 2020], and language models emit offensive text [Abid et al., 2021, Perez et al., 2022].”
BackgroundFirst Tragedy, then Parse: History Repeats Itself in the New Era of Large Language Models
Saphra, N., Fleisig, E., Cho, K., & Lopez, A. (2024) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2024.naacl-long.128 · Cited by 23
“…along with increased freedom to produce text on topics where there is real-world disagreement, including social, ethical, and political concerns (Abid et al., 2021; Blodgett et al., 2020; Liu et al., 2021; Zhao et al., 2021), mean that human evaluation issues will be pressing problems for the…”
BackgroundAligning Large Language Models with Diverse Political Viewpoints
Stammbach, D., Widmer, P., Cho, E., Gülçehre, Ç., & Ash, E. (2024) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2024.emnlp-main.412 · Cited by 23
Cantini, R., Cosenza, G., Orsino, A., & Talia, D. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-78977-9_4 · Cited by 23
“Furthermore, several other benchmarks for assessing bias in LLMs have been proposed for specific types of bias, including cognitive [20], gender-occupational [19], religion [12], and racial [30].”
BackgroundLiu, Y., Cong, T., Zhao, Z., Backes, M., Shen, Y., & Zhang, Y. (2026) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2026.3672386 · Cited by 23
“Despite their impressive performance, the outputs generated by LLMs can perpetuate harmful stereotypes [13], [14], [15], disseminate false information [16], [17], [18], [19], [20], or produce inappropriate and o ff ensive responses [21].”
Patterns of Bias: How Mainstream Media Operationalize Links between Mass Shootings and Terrorism
Dreier, S. K., Gade, E. K., Card, D., & Smith, N. A. (2022) · Political Communication · doi:10.1080/10584609.2022.2111484 · Cited by 23
“Meanwhile, racial and religious bigotry run rampant on social media platforms and yield biased algorithms (Garcia, 2016) that shape how language is modeled, analyzed and censored (Abid et al., 2021; Bender et al., 2021; ap et al., 2019; Noble, 2018).”
BackgroundTournesol: A quest for a large, secure and trustworthy database of reliable human judgmentsPreprint
Hoang, L. N., Faucon, L., Jungo, A., Volodin, S., Papuc, D., Liossatos, O., Crulis, B., Tighanimine, M., Constantin, I., Kucherenko, A., et al. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2107.07334 · Cited by 23
Using Large Language Models for Qualitative Analysis can Introduce Serious Bias
Ashwin, J., Chhabra, A., & Rao, V. (2026) · Sociological Methods & Research · doi:10.1177/00491241251338246 · Cited by 22
“A range of studies have examined bias in the context of LLMs, albeit not in the narrow technical sense we refer to here (Abid, Farooqi and Zou 2021; Ferrara 2023; Navigli, Conia and Ross 2023).”
Kirk, H. R., Vidgen, B., Röttger, P., & Hale, S. A. (2023) · arXiv · doi:10.48550/arxiv.2303.05453 · Cited by 22
“For example, there are extensive works documenting LLMs on fairness and bias [2, 115, 143, 162, 166, 191, 205, 222]; truthfulness, uncertainty, or hallucination [130, 106, 102]; robustness [208, 112]; privacy [35]; and toxicity [72, 171].”
BackgroundGeoAI – Accelerating a Virtuous Cycle between AI and Geo
Chauhan, L. P. S., & Shekhar, S. (2021) · International Conference on Contemporary Computing · doi:10.1145/3474124.3474179 · Cited by 22
“While this was just a PR disaster, the really serious concerns of AI failure are around discrimination against minorities by computer vision algorithms for face detection, unfair lending decisions made by banks, racially biased recidivism-risk scoring models used throughout the criminal justice system, or most recentl…”
BackgroundFisher, J. A. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-47655-6_5 · Cited by 22
Protected group bias and stereotypes in Large Language ModelsPreprint
Kotek, H., Sun, D. Q., Xiu, Z., Bowler, M., & Klein, C. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2403.14727 · Cited by 22
“Bias has similarly been found in social categories such as religion, race, nationality, disability, and occupation (Sap et al., 2020; Abid et al., 2021; Kirk et al., 2021; Ousidhoum et al., 2021; Venkit et al., 2022, 2023; Zhuo et al., 2023; Blodgett et al., 2021; Talat et al., 2022; Guo and…”
BackgroundAlDahoul, N., Ibrahim, H., Kaufman, A. R., Rahwan, T., & Zaki, Y. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2505.04171 · Cited by 23
Agbasiere, C. L., & Nze-Igwe, G. R. (2025) · Path of Science · doi:10.22178/pos.116-10 · Cited by 22
“Afterwards, it was discovered that GPT-3 produced extremely powerful, unfavourable prejudices about religious groups [1].”
Exploring the occupational biases and stereotypes of Chinese large language models
Jiang, L., Zhu, G., Sun, J., Cao, J., & Wu, J. X. (2025) · Scientific Reports · doi:10.1038/s41598-025-03893-w · Cited by 22
Aware Yet Biased: Investigating Emotional Reasoning and Appraisal Bias in Large Language Models
Tak, A. N., Gratch, J. S., & Scherer, K. R. (2025) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2025.3581461 · Cited by 21
“Given that LLMs have been proven to acquire and replicate certain human biases [19], [20] it is reasonable to ask if LLMs also internalize common emotional distortions, and perhaps pass 1949-3045 © 2025 IEEE.”
Morales-Muñoz, A., Bonillo, M. Á. I., Estrella-Ramón, A., & Herrada-Lores, S. (2026) · International Journal of Consumer Studies · doi:10.1111/ijcs.70173 · Cited by 24
Anoop, K. R., Gangan, M. P., Deepak, P., & Lajish, V. L. (2022) · Lecture notes in electrical engineering · doi:10.1007/978-981-19-4453-6_2 · Cited by 21
“[2] GPT-3 Religion Prompt completion, analogical reasoning, story generation —”
BackgroundInfluentialImplicit Personalization in Language Models: A Systematic Study
Jin, Z., Heil, N., Liu, J., Dhuliawala, S., Qi, Y., Schölkopf, B., Mihalcea, R. F., & Sachan, M. (2024) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2024.findings-emnlp.717 · Cited by 21
Strategic Demonstration Selection for Improved Fairness in LLM In-Context Learning
Hu, J., Liu, W., & Du, M. (2024) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2024.emnlp-main.425 · Cited by 21
Quantifying Bias in Text-to-Image Generative Models
Vice, J., Akhtar, N., Hartley, R. I., & Mian, A. S. (2025) · IEEE Transactions on Dependable and Secure Computing · doi:10.1109/tdsc.2025.3572115 · Cited by 21
“In the related literature [2], [4], [5], [30], [53], there is a gap in regard to evaluating general T2I model biases and a standardized method to address that is yet to be defined.”
LLMs Lost in Translation: M-ALERT uncovers Cross-Linguistic Safety InconsistenciesPreprint
Friedrich, F., Tedeschi, S., Schramowski, P., Brack, M., Navigli, R., Nguyen, H., Li, B., & Kersting, K. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2412.15035 · Cited by 21
“Similarly, numerous studies have found bias in the outputs of language models (Abid et al., 2021; Ganguli et al., 2023; Liang et al., 2023).”
A Comprehensive Analysis of Large Language Model Outputs: Similarity, Diversity, and BiasPreprint
Smith, B., Bouadjenek, M. R., Kheya, T. A., Dawson, P., & Aryal, S. (2025) · arXiv.org · doi:10.48550/arxiv.2505.09056 · Cited by 15
“…heavily focused on investigating and addressing such biases in LLMs, which might be present as derogatory languages towards certain minority groups [46, 47], inconsistencies in system performance across different linguistic variations [48–51], misrepresentations of groups in society [52, 53],…”
Opportunities and Risks of LLMs for Scalable Deliberation with Polis
Small, C. T., Vendrov, I., Durmus, E., Homaei, H., Barry, E., Cornebise, J., Suzman, T., Ganguli, D., & Megill, C. (2023) · arXiv · doi:10.48550/arxiv.2306.11932 · Cited by 20
Decoding Artificial Sociality: Technologies, Dynamics, Implications
Depounti, I., & Natale, S. (2025) · New Media & Society · doi:10.1177/14614448251359217 · Cited by 20
“Generative AI models have been repeatedly found to exhibit bias (Abid et al., 2021; Currie et al., 2024; Hu et al., 2024; Kotek et al., 2023; Rotaru et al., 2024; Wyer and Black, 2025) in relation to culture, gender, class, race, political views, and religion, and have been critiqued for operating…”
Artificial intelligence and international conflict in cyberspace
Cristiano, F., Broeders, D. W. G. A., Delerue, F., Douzet, F., & Géry, A. (2023) · Routledge eBooks · doi:10.4324/9781003284093-1 · Cited by 20
Halim, S. M., Irtiza, S., Hu, Y., Khan, L. R., & Thuraisingham, B. M. (2023) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn54540.2023.10191114 · Cited by 20
“Research [3], [26] shows that language models like GPT-2 often have inherent biases in them, causing them to generate offensive responses targeted towards certain minorities.”
BackgroundUnderstanding and evaluating harms of AI-generated image captions in political images
Sarhan, H., & Hegelich, S. (2023) · Frontiers in Political Science · doi:10.3389/fpos.2023.1245684 · Cited by 21
A comprehensive analysis of gender, racial, and prompt-induced biases in large language models
Torres, N., Ulloa, C., Araya, I., Ayala, M., & Jara, S. (2024) · International Journal of Data Science and Analytics · doi:10.1007/s41060-024-00696-6 · Cited by 20
“In addition, Abid et al. [12] showed that GPT-3 exhibited persistent anti-Muslim bias, associating Muslims with violent concepts more frequently than other religious groups.”
Search and Society: Reimagining Information Access for Radical Futures
Mitra, B. (2025) · Information Retrieval Research · doi:10.54195/irrj.19654 · Cited by 22
“…models reproduce, and even amplify, harmful stereotypes and biases of moral import (Friedman and Nissenbaum, 1996) that are present in their training data (Bolukbasi et al., 2016; Caliskan et al., 2017; Gonen and Goldberg, 2019; Blodgett et al., 2020; Bender et al., 2021; Abid et al., 2021).”
Braess’s Paradox of Generative AI
Taitler, B., & Ben-Porat, O. (2025) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v39i13.33548 · Cited by 20
Gender Bias in Text-to-Video Generation Models: A Case Study of Sora
Nadeem, M., Sohail, S. S., Cambria, E., Schuller, B. W., & Hussain, A. (2025) · IEEE Intelligent Systems · doi:10.1109/mis.2025.3561475 · Cited by 20
“10 For example, Abid et al. 9 found that Generative Pre-trained Transformer 3 (GPT-3) frequently associated Muslims with violence.”
The Ethics of Computational Social Science
Leslie, D. (2023) · Springer eBooks · doi:10.1007/978-3-031-16624-2_4 · Cited by 20
The Risks of Machine Learning SystemsPreprint
Tan, S., Taeihagh, A., & Baxter, K. K. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2204.09852 · Cited by 20
“While such datasets satiate increasingly large and data-hungry neural networks, they often contain noisy labels [117], harmful stereotypes [2, 12, 54], and even pornographic content [166].”
BackgroundAdding guardrails to advanced chatbotsPreprint
Wang, Y., & Singh, L. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2306.07500 · Cited by 20
“Researchers have shown that the generated text for the text completion task contains demographic biases when certain types of text are input by the users [17, 18].”
BackgroundFrontier AI Ethics: Anticipating and Evaluating the Societal Impacts of Language Model AgentsPreprint
Lazar, S. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2404.06750 · Cited by 20
“21 See e.g. Abid et al., 2021.”
Evidence for Hypodescent in Visual Semantic AI
Wolfe, R., Banaji, M. R., & Caliskan, A. (2022) · 2022 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3531146.3533185 · Cited by 19
“Social biases related to gender [7], age [37], religion [1], and sexuality [63] have been observed in AI systems.”
BackgroundGenerative AI and Language Diversity: Implications for Teachers and Learners
Hamad M Albeihi, H., & F. Rice, M. (2025) · Arab World English Journal · doi:10.24093/awej/vol16no1.3 · Cited by 19
“They often collaborate with linguists on different projects (Abid et al., 2021).”
Fairness and bias in AI: a sociotechnical perspective
Mimouni, S. E., & Bouhdadi, M. (2025) · Journal of Information Communication and Ethics in Society · doi:10.1108/jices-12-2024-0182 · Cited by 19
Towards fair decision: A novel representation method for debiasing pre-trained models
He, J., Lin, N., Bai, Q., Liang, H., Zhou, D. M., & Yang, A. (2024) · Decision Support Systems · doi:10.1016/j.dss.2024.114208 · Cited by 19
Fairframe: a fairness framework for bias detection and mitigation in news
Sallami, D., & Aïmeur, E. (2024) · AI and Ethics · doi:10.1007/s43681-024-00568-6 · Cited by 19
“Moreover, emerging research has expanded the understanding of bias beyond simple demographic factors, investigating how biases related to race, gender, disability, nationality, and religion are replicated in NLP models [40–42].”
Ecosystem Graphs: Documenting the Foundation Model Supply Chain
Bommasani, R., Soylu, D., Liao, T. I., Creel, K., & Liang, P. (2024) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1609/aies.v7i1.31629 · Cited by 20
Algorithmic bias: sexualized violence against women in GPT-3 models
Wyer, S., & Black, S. (2025) · AI and Ethics · doi:10.1007/s43681-024-00641-0 · Cited by 19
“Topic 8 relates to Muslims which has been highly documented [ 1], yet interestingly the word ‘hijab’ features highly in this topic providing a gendered aspect within this topic (see appendix A).”
Mind the data gap(s): Investigating power in speech and language datasets
Markl, N. (2022) · Workshop on Language Technology for Equality, Diversity, Inclusion (LT-EDI) · doi:10.18653/v1/2022.ltedi-1.1 · Cited by 19
“…groups are often misrepresented in text data drawn from the internet (see also Tripodi, 2021; Sun and Peng, 2021), which can lead to the reproduction of harmful stereotypes and dominant ideologies (such as islamophobia), further entrench-ing their marginalised position (Abid et al., 2021).”
BackgroundDiversity in Computer Science: Design Artefacts for Equity and Inclusion
Bjørn, P., Menéndez-Blanco, M., & Borsotti, V. (2022) · Springer eBooks · doi:10.1007/978-3-031-13314-5 · Cited by 20
Exploring the Nexus of Large Language Models and Legal Systems: A Short SurveyPreprint
Qin, W., & Sun, Z. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2404.00990 · Cited by 19
From Word to World: Evaluate and Mitigate Culture Bias in LLMs via Word Association Test
Dai, X., Zhou, L., Wang, B., & Li, H. (2025) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2025.emnlp-main.1246 · Cited by 19
Artificial intelligence in hybrid and information warfare
Moy, W. R., & Gradoń, K. (2023) · Routledge eBooks · doi:10.4324/9781003284093-4 · Cited by 18
Exploring the Effect of Multiple Natural Languages on Code Suggestion Using GitHub Copilot
Koyanagi, K., Wang, D., Noguchi, K., Kondo, M., Serebrenik, A., Kamei, Y., & Ubayashi, N. (2024) · International Conference on Mining Software Repositories (MSR) · doi:10.1145/3643991.3644917 · Cited by 18
“Regarding religion bias, Abid et al. [1] investigated the anti-Muslim bias, demonstrating that it appears consistently and creatively in different uses of the model and that it is severe even compared to biases about other religious groups.”
BackgroundParolin, E. S., Hosseini, M., Hu, Y., Khan, L. R., Brandt, P. T., Osorio, J., & D'Orazio, V. J. (2022) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1145/3514094.3534178 · Cited by 18
“Language models such as those used here have been shown to encode biases from the training data (as discussed in previous work [1, 17, 24, 27, 58, 79, 80]).”
MethodArtificial Intelligence and International Conflict in Cyberspace
Broeders, D. W. G. A., Cristiano, F., Delerue, F., Douzet, F., & Géry, A. (2023) · Routledge eBooks · doi:10.4324/9781003284093 · Cited by 18
A Roadmap for Big ModelPreprint
Yuan, S., Zhao, H., Zhao, S., Leng, J., Liang, Y., Wang, X., Yu, J., Lv, X., Shao, Z., He, J., et al. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2203.14101 · Cited by 18
Fair and Optimal Classification via Post-ProcessingPreprint
Xian, R., Yin, L., & Zhao, H. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2211.01528 · Cited by 18
“crawled from the internet) on which the language models are pre-trained may contain historical social bias, and empirical investigations have shown that such bias could be propagated and amplified in downstream applications (Bolukbasi et al., 2016; Zhao et al., 2018; Abid et al., 2021).”
BackgroundWhat's in the Box? A Preliminary Analysis of Undesirable Content in the Common Crawl Corpus
Luccioni, A. S., & Viviano, J. D. (2021) · arXiv · doi:10.48550/arxiv.2105.02732 · Cited by 17
“…larger corpora generally result in better models (Kaplan et al., 2020; Sun et al., 2017), data quality and corpora content also plays a major role in the caliber and appropriateness of these models for the various downstream applications (Florez, 2019; Abid et al., 2021; Bhardwaj et al., 2021).”
BackgroundBeauty and the Bias: Exploring the Impact of Attractiveness on Multimodal Large Language Models
Gulati, A., Moreno D’Incà, Sebe, N., Lepri, B., & Oliver, N. (2025) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1609/aies.v8i2.36619 · Cited by 18
“In fact, a variety of social biases [Yeh et al.(2023)] and biases related to reasoning and decision-making [Itzhak et al.(2024)] have been reported in LLMs, including sentiment [Huang et al.(2020)], religious [Abid et al.(2021)] and stereotype [Nadeem et al.(2021)] biases.”
Banerjee, A., & Banik, D. (2025) · ACM Computing Surveys · doi:10.1145/3773076 · Cited by 18
“Notwithstanding the advancements, there remain legal concerns regarding the use of LLMs in the legal field, such as copyright violations, privacy breaches [61], and discriminatory and biassed practices [1].”
Homophily-induced emergence of biased structures in LLM-based multi-agent AI systems
Mehdizadeh, A., & Hilbert, M. (2025) · Social Network Analysis and Mining · doi:10.1007/s13278-025-01535-7 · Cited by 17
“For example, evaluations reveal that models often associate specific religious or ethnic identities with violent or negative terms [87].”
Han, H. (2024) · Ethics & Behavior · doi:10.1080/10508422.2024.2347661 · Cited by 17
“…society and generate ethically problematic predictions, such as the generation of hate and discriminatory speeches based on the probabilistic prediction of the prevalence of such expressions in the real world (Abid et al., 2021; Naous et al., 2023; Navigli et al., 2023; Srivastava et al., 2023).”
Relation labeling in product knowledge graphs with large language models for e-commerce
Chen, J., Ma, L., Li, X., Xu, J., Cho, J. H. D., Nag, K., Körpeoğlu, E., Kumar, S., & Achan, K. (2024) · International Journal of Machine Learning and Cybernetics · doi:10.1007/s13042-024-02274-5 · Cited by 17
Skin-in-the-Game: Decision Making via Multi-Stakeholder Alignment in LLMs
Sel, B., Shanmugasundaram, P., Kachuee, M., Zhou, K., Jia, R., & Jin, M. (2024) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2024.acl-long.751 · Cited by 17
Racial Steering by Large Language Models: A Prospective Audit of GPT-4 on Housing Recommendations
Liu, E. J., So, W., Hosoi, P., & D’Ignazio, C. (2024) · ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization (EAAMO) · doi:10.1145/3689904.3694709 · Cited by 17
“In an audit study on religion, Abid et al. [4] show that interactions involving the word “Muslim” with GPT-3 spurred more violence-associated content compared to inputs referencing other religious groups.”
Mushtaq, A., Naeem, R., Taj, I., Ghaznavi, I., & Qadir, J. (2025) · IEEE Global Engineering Education Conference (EDUCON) · doi:10.1109/educon62633.2025.11016405 · Cited by 18
“Many LLMs are trained on predominantly secular, materialistic, and reductionist datasets and benchmarks [11], which can inadvertently lead them to reflect biases that marginalize or overlook diverse cultural, ethical, and spiritual perspectives, especially those outside Western paradigms.”
Jo, J., Zhang, H., Cai, J., & Goyal, N. (2025) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3715275.3732077 · Cited by 17
“However, this warrants caution, since AI can replicate or even amplify existing biases [1].”
WorldView-Bench: A Benchmark for Evaluating Global Cultural Perspectives in Large Language Models
Mushtaq, A., Taj, I., Naeem, R., Ghaznavi, I., & Qadir, J. (2026) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.19001 · Cited by 17
Flexible text generation for counterfactual fairness probing
Fryer, Z., Axelrod, V., Packer, B., Beutel, A., Chen, J., & Webster, K. (2022) · Workshop on Online Abuse and Harms (WOAH) · doi:10.18653/v1/2022.woah-1.20 · Cited by 17
Identifying Gender Bias in Generative Models for Mental Health Synthetic Data
Lozoya, D. C., D’Alfonso, S., & Conway, M. (2023) · IEEE International Conference on Healthcare Informatics (ICHI) · doi:10.1109/ichi57859.2023.00109 · Cited by 19
“Religious biases in large language models (LLMs) have also been addressed; in [21] they showed that GPT-3 persistently associated Muslims with violence.”
BackgroundChen, L., Preece, D. A., Sikka, P., Gross, J. J., & Krause, B. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2407.11387 · Cited by 17
“Another key issue is the model’s tendency to produce content that seems plausible but is factually inaccurate or irrelevant, a phenomenon commonly referred to as “hallucination” [42, 49, 5].”
BackgroundBuilding Safe GenAI Applications: An End-to-End Overview of Red Teaming for Large Language ModelsPreprint
Purpura, A., Wadhwa, S., Zymet, J., Gupta, A., Luo, A., Rad, M. K., Shinde, S. D., & Sorower, M. S. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2503.01742 · Cited by 17
“Such attacks could provoke LLMs to generate misinformative, biased, or toxic content (Abid et al., 2021; Lin et al., 2022) or expose private information (Car-lini et al., 2021).”
Metadata
- Authors
- Abid, A., Farooqi, M., & Zou, J.
- Affiliations
- Stanford University (US)
- McMaster University (CA)
- Venue
- Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society
- Publisher
- ACM
- Published
- 2021-07-21
- Pages
- 298-306
- ISSN
- 3065-8365 (ISSN-L)
- OpenAlex
- W3184144760
- Cited by
- 476
- References
- 3
- Citations by year
- 2026: 81 · 2025: 124 · 2024: 105 · 2023: 97 · 2022: 36 · 2021: 33
- FWCI
- 61.36
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- Proceedings of the AAAI/ACM Conference on AI Ethics and Society (publishedVersion)
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Artificial Intelligence › Hate Speech and Cyberbullying Detection
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Keywords
- Islamophobia, religious bias, large language models, GPT-3, analogical reasoning, adversarial prompts, story generation, bias mitigation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.