The DOI matches its Crossref record; 10 of the 10 references it cites were found in a registry or an index.
Persistent Anti-Muslim Bias in Large Language Models
Open accessAbstract
It has been observed that large-scale language models capture undesirable societal biases, e.g. relating to race and gender; yet religious bias has been relatively unexplored. We demonstrate that GPT-3, a state-of-the-art contextual language model, captures persistent Muslim-violence bias. We probe GPT-3 in various ways, including prompt completion, analogical reasoning, and story generation, to understand this anti-Muslim bias, demonstrating that it appears consistently and creatively in different uses of the model and that it is severe even compared to biases about other religious groups. For instance, Muslim is analogized to terrorist in 23% of test cases, while Jewish is mapped to its most common stereotype, money, in 5% of test cases. We quantify the positive distraction needed to overcome this bias with adversarial text prompts, and find that use of the most positive 6 adjectives reduces violent completions for Muslims from 66% to 20%, but which is still higher than for other religious groups.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Pre-trained on massive amounts of text on the Internet, LMs are known to produce linguistically fawed, factually incorrect, or even ethically problematic text at times [1,24].”
From CoAuthor: Designing a Human-AI Collaborative Writing Dataset for Exploring Language Model Capabilities (2022) · cites this paper as [1]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“LLMs and other AI tools may have a number of biases that could arise from various factors, such as unbalanced training datasets74–77.”
From On opportunities and challenges of large multimodal foundation models in education (2025) · cites this paper as [75]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“it is also a liability; all AI systems might inherit the same problematic biases of a few foundation models [Bolukbasi et al. 2016; Caliskan et al. 2017; Abid et al. 2021, inter alia]”
From On the Opportunities and Risks of Foundation Models (2021) · cites this paper as Abid et al. 2021
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Previous work has shown that language models tend to exhibit human-like biases with respect to specific protected groups such as gender, ethnicity or religious orientation3–6.”
From Generative language models exhibit social identity biases (2024) · cites this paper as [4]
In the citing paper
“If a medical LLM has been trained on data that contains misinformation or biased content, LLMs deployed for clinical decision-making are at risk of reproducing these problematic contents [120,124–126].”
From Bias in medical AI: Implications for clinical decision-making (2024) · cites this paper as [126]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404 · cited 1×
“These languagemodels have evolved, from learning individual word vectors with single-layer models [8], to more complex language generation architectures such as recurrent neural networks [5] and most recently transformers [4, 6, 14].”
BackgroundEfficient Estimation of Word Representations in Vector Space
Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1301.3781 · Cited by 26,568 · cited 1×
“These languagemodels have evolved, from learning individual word vectors with single-layer models [8], to more complex language generation architectures such as recurrent neural networks [5] and most recently transformers [4, 6, 14].”
BackgroundLanguage Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372 · cited 4×
“These languagemodels have evolved, from learning individual word vectors with single-layer models [8], to more complex language generation architectures such as recurrent neural networks [5] and most recently transformers [4, 6, 14].”
BackgroundMethodInfluentialSelf-Critical Sequence Training for Image Captioning
Rennie, S. J., Marcheret, E., Mroueh, Y., Ross, J., & Goel, V. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.131 · Cited by 2,112 · cited 1×
“These descriptions are generated by passing images through a pretrained image captioning model [11], and fine-tuning GPT-3 to transform the short captions into long descriptions, as shown in Fig.”
MethodReformer: The Efficient Transformer
Kitaev, N., Kaiser, Ł., & Levskaya, A. (2020) · arXiv · doi:10.48550/arxiv.2001.04451 · Cited by 1,435 · cited 1×
“These languagemodels have evolved, from learning individual word vectors with single-layer models [8], to more complex language generation architectures such as recurrent neural networks [5] and most recently transformers [4, 6, 14].”
BackgroundGradio: Hassle-Free Sharing and Testing of ML Models in the WildPreprint
Abid, A., Abdalla, A., Ali, A., Khan, D., Alfozan, A., & Zou, J. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1906.02569 · Cited by 434 · cited 1×
“We created a visual interface using the Gradio library [1], and used the model to generate captions for stock photos.”
MethodSelf-citationIdentifying and Reducing Gender Bias in Word-Level Language Models
Bordia, S., & Bowman, S. R. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Student Research Workshop · doi:10.18653/v1/n19-3002 · Cited by 219 · cited 1×
“Previous work has explored the biases that are ultimately learned by language models, with the primary focus being racial and gender stereotypes [2, 3, 7, 12].”
BackgroundThe Woman Worked as a Babysitter: On Biases in Language GenerationPreprint
Sheng, E., Chang, K.-W., Natarajan, P., & Peng, N. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.01326 · Cited by 203 · cited 2×
“In [12], the authors made similar observations about language generation models.”
BackgroundCombining Independent Modules to Solve Multiple-choice Synonym and Analogy ProblemsPreprint
Turney, P. D., Littman, M. L., Bigham, J. P., & Shnayder, V. (2003) · arXiv (Cornell University) · doi:10.48550/arxiv.cs/0309035 · Cited by 139
(2020) · Lecture notes in computer science · doi:10.1007/978-3-030-62077-6 · Cited by 30
Research works printed but not linked: 6
- Tolga Bolukbasi , Kai-Wei Chang , James Y Zou , Venkatesh Saligrama , and Adam T Kalai . 2016. Man is to computer programmer as woman is to homemaker? debiasing word embeddings. Advances in neural information processing systems , Vol. 29 ( 2016 ), 4349--4357. Tolga Bolukbasi, Kai-Wei Chang, James Y Zou, Venkatesh Saligrama, and Adam T Kalai. 2016. Man is to computer programmer as woman is to homemaker? debiasing word embeddings. Advances in neural information processing systems , Vol. 29 (2016), 4349--4357.
- Andrew M Dai and Quoc V Le. 2015. Semi-supervised sequence learning. Advances in neural information processing systems , Vol. 28 ( 2015 ), 3079--3087. Andrew M Dai and Quoc V Le. 2015. Semi-supervised sequence learning. Advances in neural information processing systems , Vol. 28 (2015), 3079--3087.
- Kaiji Lu , Piotr Mardziel , Fangjing Wu , Preetam Amancharla , and Anupam Datta . 2020. Gender bias in neural natural language processing . In Logic, Language, and Security . Springer , 189--202. Kaiji Lu, Piotr Mardziel, Fangjing Wu, Preetam Amancharla, and Anupam Datta. 2020. Gender bias in neural natural language processing. In Logic, Language, and Security . Springer, 189--202.
- Moin Nadeem , Anna Bethke , and Siva Reddy . 2020. StereoSet: Measuring stereotypical bias in pretrained language models. arXiv preprint arXiv:2004.09456 ( 2020 ). Moin Nadeem, Anna Bethke, and Siva Reddy. 2020. StereoSet: Measuring stereotypical bias in pretrained language models. arXiv preprint arXiv:2004.09456 (2020). · arXiv
- Yusu Qian , Urwa Muaz , Ben Zhang , and Jae Won Hyun . 2019. Reducing gender bias in word-level language models with a gender-equalizing loss function. arXiv preprint arXiv:1905.12801 ( 2019 ). Yusu Qian, Urwa Muaz, Ben Zhang, and Jae Won Hyun. 2019. Reducing gender bias in word-level language models with a gender-equalizing loss function. arXiv preprint arXiv:1905.12801 (2019). · arXiv
- Eric Wallace , Shi Feng , Nikhil Kandpal , Matt Gardner , and Sameer Singh . 2019. Universal adversarial triggers for attacking and analyzing NLP. arXiv preprint arXiv:1908.07125 ( 2019 ). Eric Wallace, Shi Feng, Nikhil Kandpal, Matt Gardner, and Sameer Singh. 2019. Universal adversarial triggers for attacking and analyzing NLP. arXiv preprint arXiv:1908.07125 (2019). · arXiv
Cited by
On the Opportunities and Risks of Foundation Models
Bommasani, R., Hudson, D. A., Adeli, E., Altman, R., Arora, S., von Arx, S., Bernstein, M. S., Bohg, J., Bosselut, A., Brunskill, E., et al. (2021) · arXiv · doi:10.48550/arxiv.2108.07258 · Cited by 2,676
PaLM: Scaling Language Modeling with Pathways
Chowdhery, A., Narang, S., Devlin, J., Bosma, M., Mishra, G., Roberts, A., Barham, P., Chung, H. W., Sutton, C., Gehrmann, S., et al. (2022) · arXiv · doi:10.48550/arxiv.2204.02311 · Cited by 2,590
“In particular we see in Table 37 that Islam is associated with highly charged terms and stereotypes such as terrorist, violent and radical, similar to other evaluations of decoder-only models revealing anti-Muslim bias (Abid et al., 2021).”
ResultGPTs are GPTs: An Early Look at the Labor Market Impact Potential of Large Language Models
Eloundou, T., Manning, S., Mishkin, P., & Rock, D. (2023) · arXiv · doi:10.48550/arxiv.2303.10130 · Cited by 2,330
“…partly because out-of-the-box general-purpose LLMs may continue to be unreliable for various tasks due to issues such as factual inaccuracies, inherent biases, privacy concerns, and disinformation risks (Abid et al., 2021; Schramowski et al., 2022; Goldstein et al., 2023; OpenAI, 2023a).”
BackgroundA Survey of Large Language Models
Zhao, W. X., Zhou, K., Li, J., Tang, T., Dong, Z., Hou, Y., Zhang, B., Min, Y., Zhang, J., Liu, P., et al. (2026) · Frontiers of Computer Science · doi:10.1007/s11704-026-60308-3 · Cited by 2,093
Evaluating Large Language Models Trained on Code
Chen, M., Tworek, J., Jun, H., Yuan, Q., Pinto, H. P. D. O., Kaplan, J., Edwards, H., Burda, Y., Joseph, N., Brockman, G., et al. (2021) · arXiv · doi:10.48550/arxiv.2107.03374 · Cited by 1,436
“Mirroring what has been found in the case of other language models trained on Internet data (Bender et al., 2021; Blod-gett et al., 2020; Abid et al., 2021; Brown et al., 2020”
Background
Metadata
- Authors
- Abid, A., Farooqi, M., & Zou, J.
- Affiliations
- Stanford University (US)
- McMaster University (CA)
- Venue
- Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society
- Publisher
- ACM
- Published
- 2021-07-21
- Pages
- 298-306
- ISSN
- 3065-8365 (ISSN-L)
- OpenAlex
- W3184144760
- Cited by
- 476
- References
- 3
- Citations by year
- 2026: 81 · 2025: 124 · 2024: 105 · 2023: 97 · 2022: 36 · 2021: 33
- FWCI
- 61.36
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- Proceedings of the AAAI/ACM Conference on AI Ethics and Society (publishedVersion)
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Artificial Intelligence › Hate Speech and Cyberbullying Detection
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Keywords
- Islamophobia, religious bias, large language models, GPT-3, analogical reasoning, adversarial prompts, story generation, bias mitigation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Persistent Anti-Muslim Bias in Large Language Models. https://app.cochat.ai/papers/10.1145/3461702.3462624
- The paper
- Abid, A., Farooqi, M., & Zou, J. (2021). Persistent Anti-Muslim Bias in Large Language Models. Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society. https://doi.org/10.1145/3461702.3462624
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.