Persistent Anti-Muslim Bias in Large Language Models
Open accessThe DOI matches its Crossref record; 13 of the 13 references it cites were found in a registry or an index.
Abstract
It has been observed that large-scale language models capture undesirable societal biases, e.g. relating to race and gender; yet religious bias has been relatively unexplored. We demonstrate that GPT-3, a state-of-the-art contextual language model, captures persistent Muslim-violence bias. We probe GPT-3 in various ways, including prompt completion, analogical reasoning, and story generation, to understand this anti-Muslim bias, demonstrating that it appears consistently and creatively in different uses of the model and that it is severe even compared to biases about other religious groups. For instance, Muslim is analogized to terrorist in 23% of test cases, while Jewish is mapped to its most common stereotype, money, in 5% of test cases. We quantify the positive distraction needed to overcome this bias with adversarial text prompts, and find that use of the most positive 6 adjectives reduces violent completions for Muslims from 66% to 20%, but which is still higher than for other religious groups.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Pre-trained on massive amounts of text on the Internet, LMs are known to produce linguistically fawed, factually incorrect, or even ethically problematic text at times [1,24].”
From CoAuthor: Designing a Human-AI Collaborative Writing Dataset for Exploring Language Model Capabilities (2022) · cites this paper as [1]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“LLMs and other AI tools may have a number of biases that could arise from various factors, such as unbalanced training datasets74–77.”
From On opportunities and challenges of large multimodal foundation models in education (2025) · cites this paper as [75]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“it is also a liability; all AI systems might inherit the same problematic biases of a few foundation models [Bolukbasi et al. 2016; Caliskan et al. 2017; Abid et al. 2021, inter alia]”
From On the Opportunities and Risks of Foundation Models (2021) · cites this paper as Abid et al. 2021
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Previous work has shown that language models tend to exhibit human-like biases with respect to specific protected groups such as gender, ethnicity or religious orientation3–6.”
From Generative language models exhibit social identity biases (2024) · cites this paper as [4]
In the citing paper
“If a medical LLM has been trained on data that contains misinformation or biased content, LLMs deployed for clinical decision-making are at risk of reproducing these problematic contents [120,124–126].”
From Bias in medical AI: Implications for clinical decision-making (2024) · cites this paper as [126]
1 more claim about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404 · cited 1×
“These languagemodels have evolved, from learning individual word vectors with single-layer models [8], to more complex language generation architectures such as recurrent neural networks [5] and most recently transformers [4, 6, 14].”
BackgroundEfficient Estimation of Word Representations in Vector Space
Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1301.3781 · Cited by 26,568 · cited 1×
“These languagemodels have evolved, from learning individual word vectors with single-layer models [8], to more complex language generation architectures such as recurrent neural networks [5] and most recently transformers [4, 6, 14].”
BackgroundLanguage Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372 · cited 4×
“These languagemodels have evolved, from learning individual word vectors with single-layer models [8], to more complex language generation architectures such as recurrent neural networks [5] and most recently transformers [4, 6, 14].”
BackgroundMethodInfluentialSelf-Critical Sequence Training for Image Captioning
Rennie, S. J., Marcheret, E., Mroueh, Y., Ross, J., & Goel, V. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.131 · Cited by 2,112 · cited 1×
“These descriptions are generated by passing images through a pretrained image captioning model [11], and fine-tuning GPT-3 to transform the short captions into long descriptions, as shown in Fig.”
MethodReformer: The Efficient Transformer
Kitaev, N., Kaiser, Ł., & Levskaya, A. (2020) · arXiv · doi:10.48550/arxiv.2001.04451 · Cited by 1,435 · cited 1×
“These languagemodels have evolved, from learning individual word vectors with single-layer models [8], to more complex language generation architectures such as recurrent neural networks [5] and most recently transformers [4, 6, 14].”
BackgroundSemi-supervised Sequence Learning
Dai, A. M., & Le, Q. V. (2015) · arXiv · doi:10.48550/arxiv.1511.01432 · Cited by 994
StereoSet: Measuring stereotypical bias in pretrained language models
Nadeem, M., Bethke, A., & Reddy, S. (2020) · arXiv · doi:10.48550/arxiv.2004.09456 · Cited by 643 · cited 2×
“As a result, even though the various language models have different architectures, since they are trained on similar corpora of text, they often learn similar biases [9].”
BackgroundUniversal Adversarial Triggers for Attacking and Analyzing NLP
Wallace, E., Feng, S., Kandpal, N., Gardner, M., & Singh, S. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1221 · Cited by 642
Identifying and Reducing Gender Bias in Word-Level Language Models
Bordia, S., & Bowman, S. R. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Student Research Workshop · doi:10.18653/v1/n19-3002 · Cited by 219 · cited 1×
“Previous work has explored the biases that are ultimately learned by language models, with the primary focus being racial and gender stereotypes [2, 3, 7, 12].”
BackgroundThe Woman Worked as a Babysitter: On Biases in Language GenerationPreprint
Sheng, E., Chang, K.-W., Natarajan, P., & Peng, N. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.01326 · Cited by 205 · cited 2×
“In [12], the authors made similar observations about language generation models.”
Background
Research works printed but not linked: 6
- 7. T. Bolukbasi, K.-W. Chang, J. Y. Zou, V. Saligrama, A. T. Kalai, Advances in neural information processing systems 29 , 4349 (2016).
- 9. S. Bordia, S. R. Bowman, arXiv preprint arXiv:1904.03035 (2019). · arXiv
- 2. A. M. Dai, Q. V. Le, Advances in neural information processing systems 28 , 3079 (2015).
- 10. K. Lu, P. Mardziel, F. Wu, P. Amancharla, A. Datta, Logic, Language, and Security (Springer, 2020), pp. 189-202.
- 3. A. Vaswani, et al. , Advances in neural information processing systems (2017), pp. 59986008.
- 14. E. Wallace, S. Feng, N. Kandpal, M. Gardner, S. Singh, arXiv preprint arXiv:1908.07125 (2019). · arXiv
Cited by
Items 501–600 of 986
Kot, S., Engineer, M., Gilman, E., Flathmann, C., Pradhan, A., & Dixon, E. (2026) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3773029 · Cited by 8
“Typical discussions of representation by AI center on community-level representation, such as how generative AI can encode several harmful stereotypes 4:30 S. Kot et al. [119] and assumptions pertaining to gender [2], race [144], religion [1], and abilities [69].”
Injecting Bias through Prompts: Analyzing the Influence of Language on LLMs
Torres, N., Ulloa, C., Araya, I., Ayala, M., & Jara, S. (2024) · International Conference of the Chilean Computer Science Society (SCCC) · doi:10.1109/sccc63879.2024.10767653 · Cited by 8
“In the context of LLMs, Abid et al. [1] showed that GPT-3 exhibited persistent anti-Muslim bias, associating Muslims with violent concepts more frequently than other religious groups.”
Cook, R. A., Lalor, J. P., & Abbasi, A. (2025) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2025.naacl-long.129 · Cited by 8
“…current Large Language Model (LLM) paradigm, the addition of complexity-based techniques risks exacerbating existing data biases from pretraining data content (Li et al., 2020; Abid et al., 2021), human annotation (Kirk et al., 2023; Gururangan et al., 2018), and user feedback (Qiu et al., 2022).”
WSC+: Enhancing The Winograd Schema Challenge Using Tree-of-Experts
Zahraei, P. S., & Emami, A. (2024) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2024.eacl-long.99 · Cited by 8
Equal Confusion Fairness: Measuring Group-Based Disparities in Automated Decision Systems
Gürsoy, F., & Kakadiaris, I. A. (2022) · IEEE ... International Conference on Data Mining workshops · doi:10.1109/icdmw58026.2022.00027 · Cited by 8
Feldman, P., Dant, A., Foulds, J. R., & Pan, S. (2022) · arXiv (Cornell University) · doi:10.13016/m2z78y-9jth · Cited by 8
Survey of Generative Methods for Social Media AnalysisPreprint
Matwin, S., Milios, A., Prałat, P., Soares, A., & Théberge, F. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2112.07041 · Cited by 8
“Figure 13: Persistent anti-Muslim bias in GPT-3 [1]”
BackgroundMore Than Words: Towards Better Quality Interpretations of Text ClassifiersPreprint
Zafar, M. B., Schmidt, P., Donini, M., Archambeau, C., Bießmann, F., Das, S. R., & Kenthapadi, K. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2112.12444 · Cited by 8
“Recent work has highlighted several other issues like environmental impact and bias [1, 9, 17, 54].”
BackgroundLearning Multiplex Representations on Text-Attributed Graphs with One Language Model EncoderPreprint
Jin, B., Zhang, W., Zhang, Y., Yu, M., Zhao, H., & Han, J. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2310.06684 · Cited by 8
“However, some researchers have pointed out certain limitations associated with these models, including the presence of social bias (Liang et al., 2021) and the propagation of misinformation (Abid et al., 2021).”
BackgroundMarkelius, A. (2025) · Oxford University Press eBooks · doi:10.1093/9780198945215.003.0013 · Cited by 8
Islamophobic Artificial Intelligence in the USA: A Critical Analysis of Religious Bias in DatasetsPreprint
Munir, B. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5265355 · Cited by 8
Social Group Bias in AI FinancePreprint
Cook, T. R., & Kazinnik, S. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5287153 · Cited by 8
The artefacts of intelligence: governing scientists' contribution to AI proliferation
Shevlane, T. (2023) · Oxford University Research Archive (ORA) (University of Oxford) · doi:10.5287/ora-rdvmqvvad · Cited by 8
Generative Monoculture in Large Language Models
Wu, F., Black, E., & Chandrasekaran, V. (2024) · arXiv · doi:10.48550/arxiv.2407.02209 · Cited by 7
“Recent evidence has shown that LLMs have dangerous tendencies: they convincingly return incorrect information [5, 6, 7], produce toxic language [8, 9], and can effectively propagate misinformation [10, 11].”
BackgroundAgainst AI Jurisprudence: Large Language Models and the False Promises of Empirical Judging
Pruss, D., & Allen, J. (2025) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1609/aies.v8i3.36695 · Cited by 7
Heyn, H.-M., Knauss, E., Malleswaran, I., & Dinakaran, S. (2024) · Requirements Engineering · doi:10.1007/s00766-024-00415-4 · Cited by 7
“Racial, religious, or gender biases are introduced through a lack of insight into the (sometimes immensely large set of) training data and missing runtime checks for example in large language models such as GPT-3 [1], or facial recognition software based on deep learning [57].”
REFINE-LM: Mitigating Language Model Stereotypes via Reinforcement Learning
Qureshi, M. R., Es-Sebbani, N., Galárraga, L. A., Graham, Y., Couceiro, M., & Bouraoui, Z. (2024) · Frontiers in artificial intelligence and applications · doi:10.3233/faia240970 · Cited by 7
STOP! Benchmarking Large Language Models with Sensitivity Testing on Offensive Progressions
Morabito, R., Madhusudan, S., McDonald, T., & Emami, A. (2024) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2024.emnlp-main.243 · Cited by 7
“…shown to disproportionately impact individuals of specific social demographics, such as religion, sex, race, age, educational institution, nationality, and disability (Abid et al., 2021; Gonen and Goldberg, 2019; Wan et al., 2023; Sap et al., 2021; Kamruzzaman et al., 2024; Venkit et al., 2022).”
BackgroundFine-Tuned LLMs are “Time Capsules” for Tracking Societal Bias Through Books
Madhusudan, S., Morabito, R., Reid, S., Sadr, N. G., & Emami, A. (2025) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2025.naacl-long.118 · Cited by 7
Wencker, T., Borst, J., & Niekler, A. (2025) · Evaluation · doi:10.1177/13563890251330911 · Cited by 8
“Research has identified political (Motoki et al., 2024) and religious (Abid et al., 2021) biases in large-language models.”
Pantelidou, N., Leivada, E., Montero, R., & Morosi, P. (2026) · PLoS ONE · doi:10.1371/journal.pone.0343164 · Cited by 7
“If the data are not carefully selected and preprocessed, models may produce biased, harmful, and stereotypical responses [14-16].”
Saturn Platform: Foundation Model Operations and Generative AI for Financial Services
Busson, A., Gaio, R., Rocha, R. H., Evangelista, F., Rizzi, B., Carvalho, L., Miceli, R., Rabaioli, M., & Favaro, D. (2023) · Brazilian Symposium on Multimedia and the Web · doi:10.5753/webmedia_estendido.2023.234354 · Cited by 2
“While this homogenization produces hugely high leverage (improvements to the basic models can bring immediate benefits for most of the other foundational models), all AI systems can inherit the same problematic biases from some foundational models [1].”
BackgroundTransformer-Based Language Models for Bulgarian
Marinova, I., Simov, K., & Osenova, P. (2023) · International conference Recent advances in natural language processing · doi:10.26615/978-954-452-092-2_077 · Cited by 7
“Abubakar Abid and Zou (2021) demonstrate that GPT-3 shows persistent Muslim-violence bias.”
BackgroundParra, I. (2024) · Conference of the European Chapter of the Association for Computational Linguistics: Student Research Workshop (EACL SRW) · doi:10.18653/v1/2024.eacl-srw.6 · Cited by 7
Racial Skew in Fine-Tuned Legal AI Language Models
Malic, V. Q., Kumari, A., & Liu, X. (2023) · IEEE ... International Conference on Data Mining workshops · doi:10.1109/icdmw60847.2023.00037 · Cited by 7
Relabeling Minimal Training Subset to Flip a Prediction
Yang, J., Xu, L., & Yu, L. (2024) · Findings of the Association for Computational Linguistics: EACL · doi:10.18653/v1/2024.findings-eacl.73 · Cited by 7
“They are prone to challenges including, but not limited to, social biases (Hutchinson et al., 2020; Bender et al., 2021; Abid et al., 2021; Wei-dinger et al., 2021; Bommasani et al., 2022) and the spread of misinformation (Evans et al., 2021; Lin et al., 2022).”
BackgroundÉtica na era dos Modelos de Linguagem Massivos (LLMs): um estudo de caso do ChatGPT
Figênio, M. R., & Gomes-Jr, L. (2023) · doi:10.5753/erbd.2023.229510 · Cited by 7
Loquacity and Visible Emotion: ChatGPT as a Policy AdvisorPreprint
Biancotti, C., & Camassa, C. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4533699 · Cited by 7
Exploring the Jungle of Bias: Political Bias Attribution in Language Models via Dependency AnalysisPreprint
Jenny, D. F., Billeter, Y., Sachan, M., Schölkopf, B., & Jin, Z. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2311.08605 · Cited by 7
Markelius, A. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2406.06400 · Cited by 7
“Notably, these biases manifest in various forms, including consistent conflation of Muslims with terrorism [36], as well as a marked prioritization of Israeli human rights over those of Palestinians [37].”
BackgroundExploring the Benefits of Domain-Pretraining of Generative Large Language Models for ChemistryPreprint
Acharya, A., Sharma, S. K., Cosbey, R., Subramanian, M., Howland, S., & Glenski, M. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2411.03542 · Cited by 7
“(Abid et al., 2021; Buolamwini and Gebru, 2018; Liang et al., 2021; Nadeem et al., 2021; Welbl et al., 2021) However, when used within the parameters of our experiments detailed in this paper, we did not see such behaviour from any of the models.”
MethodTowards Resource Efficient and Interpretable Bias Mitigation in Large Language ModelsPreprint
Tong, S., Zemour, E., Lu, J., Lohanimit, R., & Kagal, L. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2412.01711 · Cited by 7
“Bias in NLG negatively affect society in many ways, ranging from propagating and amplifying bias [1, 21] to discouraging certain groups from adopting the technology [30] or even increasing their vulnerability to harm and discrimination [32, 22].”
Bias, Accuracy, and Trust: Gender-Diverse Perspectives on Large Language ModelsPreprint
Gaba, A., Wall, E., Babu, T. R., Brun, Y., Hall, K. W., & Bearfield, C. X. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2506.21898 · Cited by 7
“This issue is further exacerbated by biases that extend beyond gender, such as anti-Muslim bias, which has been documented in LLMs [2].”
More of the Same: Persistent Representational Harms Under Increased RepresentationPreprint
Mickel, J., De‐Arteaga, M., Liu, L., & Tian, K. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2503.00333 · Cited by 7
“Key approaches include pre-processing techniques [31, 95, 108], prompting strategies (a specialized form of pre-processing) [1, 34, 38, 64, 100, 107], in-training methods [45, 111], such as reinforcement learning with human feedback (RLHF) [77], intra-processing techniques [48], and post-processing…”
Not Your Typical Sycophant: The Elusive Nature of Sycophancy in Large Language ModelsPreprint
Natan, S. B., & Tsur, O. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2601.15436 · Cited by 7
Wingate, D., Carty, S., Coates, J., Feldman, D., Fulda, N., Howell, L., Israelson, B., Jacobs, D., Karr, J., Kimes, J. P., et al. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2605.24319 · Cited by 7
“Abid et al. (2021) document persistent anti-Muslim bias in GPT-3; Hemmatian et al. (2023) show it persists in safety-tuned successors.”
The Alignment Problem in Context
Millière, R. (2023) · arXiv · doi:10.48550/arxiv.2311.02147 · Cited by 6
A cross-lingual syntactic investigation of gender bias and stereotyping in GPT-4o: English vs Hindi
Arora, S., & Srivastava, A. (2024) · AI and Ethics · doi:10.1007/s43681-024-00565-9 · Cited by 6
“This bias encompasses more than just gender; it extends to other social categories like religion, race, nationality, disability, and occupations [1, 29, 33, 37, 38, 42].”
RV4Chatbot: Are Chatbots Allowed to Dream of Electric Sheep?
Gatti, A., Mascardi, V., & Ferrando, A. (2024) · FMAS@iFM · doi:10.4204/eptcs.411.5 · Cited by 3
“Besides religious bias [1], gender bias and stereotypes [33], and hallucinations [48], major privacy concerns are associated with LLMs.”
Dönmez, E., Vu, T., & Faleńska, A. (2024) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2024.emnlp-main.1019 · Cited by 6
“Models are known to produce text with undesired properties such as bias, racism, and violence (Abid et al., 2021; Lucy and Bamman, 2021; Shaikh et al., 2023; Navigli et al., 2023).”
BackgroundHumanist in the Loop: Teaching Critical AI Literacies, Episode 1
Goodlad, L. M. E. (2025) · Critical AI · doi:10.1215/2834703x-11908331 · Cited by 6
Comprehensive survey of large models-driving intelligent decision making
Zheng, Y., Wu, T., Xiao, X., & Xu, Z. (2025) · Applied Soft Computing · doi:10.1016/j.asoc.2025.113524 · Cited by 6
Bypassing Guardrails: Lessons Learned from Red Teaming ChatGPT
Zhuo, T. Y., Huang, Y., Chen, C., Du, X., & Xing, Z. (2026) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3747288 · Cited by 6
“Our study also differs from prior ones that focus on representational bias [5, 37, 47, 54, 58, 66], information leakage and misuse [4, 16, 32], and ethical prompting strategies [69, 78, 87, 90].”
InfluentialWeQA: A Benchmark for Retrieval Augmented Generation in Wind Energy Domain
Meyur, R., Phan, H., Wagle, S., Strube, J., Halappanavar, M. M., Horawalavithana, S., Acharya, A., & Munikoti, S. (2025) · Workshop on NLP for Positive Impact (NLP4PI) · doi:10.18653/v1/2025.nlp4pi-1.20 · Cited by 6
“…exhibit a wide variety of bias – e.g., religious, gender, race, profession, and cultural – and frequently generate answers that are incorrect, misogynistic, antisemitic, and generally toxic (Abid et al., 2021; Buolamwini and Gebru, 2018; Liang et al., 2021; Nadeem et al., 2021; Welbl et al., 2021).”
FanFAIR: sensitive data sets semi-automatic fairness assessment
Gallese, C., Scantamburlo, T., Manzoni, L., Giannerini, S., & Nobile, M. S. (2025) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-025-03184-4 · Cited by 6
Chen, Z., Liu, C., Ren, Y., Song, G., & Zhang, X. (2025) · Computers in Human Behavior Reports · doi:10.1016/j.chbr.2025.100827 · Cited by 6
Examining Age-Bias and Stereotypes of Aging in LLMs
Dewan, S., Shaikh, I., Shaw, C., Sahoo, A. K., Jha, A., & Pradhan, A. (2025) · International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS) · doi:10.1145/3663547.3746464 · Cited by 6
“As such, understanding the types of stereotypes or assumptions encoded in these models about different underrepresented groups including gender [23], race, ethnicity [34, 35], culture [19, 36], religion [1] and ability [12] has received much research attention.”
Adapting Bias Evaluation to Domain Contexts using Generative Models
Quiroga, T., Bravo-Márquez, F., & Barrière, V. (2025) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2025.emnlp-main.1424 · Cited by 6
“However, these models have also been shown to replicate social biases, raising significant ethical concerns (Blodgett et al., 2020; Abid et al., 2021; Gallegos et al., 2024).”
Tu, Z., Cheng, C., & Du, Y. (2025) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2025.emnlp-main.933 · Cited by 6
“Despite that, cleaning and filtering techniques could help address data quality issues, even state-of-the-art models can generate incorrect answers or biased outputs (Weidinger et al., 2021; Abid et al., 2021; Ferrara, 2023).”
AI and Low-Resource Languages: Bridging the Linguistic Divide
Mullah, N. S., & Zainon, W. M. N. W. (2026) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-3373-4397-6.ch003 · Cited by 6
Cheng, Y., Moakhar, A. S., Fan, C., Hosseini, P., Faghih, K., Sodagar, Z., Wang, W., & Feizi, S. (2026) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2026.findings-acl.1848 · Cited by 6
“Alignment spans capabilities like instruction following (Zhou et al., 2023; Qin et al., 2024), harm avoidance (Chao et al., 2024), bias mitigation (Abid et al., 2021; Lucy and Bam-man, 2021), hallucination reduction (Tonmoy et al., 2024), and misuse prevention (Sandbrink, 2023).”
Virtual reality and its opportunities and risks
OECD (2024) · OECD Digital Economy Outlook · doi:10.1787/a94253ae-en · Cited by 6
Hijacking algorithmic bias: analyzing the political discourse around ChatGPT on social media
Cohen, M., & Toch, E. (2025) · Information Communication & Society · doi:10.1080/1369118x.2025.2561046 · Cited by 6
“…arises from the observation that LLMs trained on large internet datasets often replicate the biases present in their training data, which can then lead to the perpetuation of existing societal biases, including stereotypes and prejudiced viewpoints (Abid et al., 2021; Weidinger et al., 2022).”
Barbieri, U., Marsico, E., Piceci, L., Fuccio, R. D., & Cassese, F. P. (2024) · Giornale Italiano di Educazione alla Salute, Sport e Didattica Inclusiva · doi:10.32043/gsd.v8i3.1179 · Cited by 6
Stereotype and Categorical Bias Evaluation via Differential Cosine Bias Measure
Sayenju, S., Aygün, R. S., Franks, B., Johnston, S., Lee, G., & Modgil, G. (2022) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata55660.2022.10020924 · Cited by 6
Cooman, J. D. (2023) · Law Technology and Humans · doi:10.5204/lthj.2787 · Cited by 6
NLP Security and Ethics, in the Wild
Lent, H., Galinkin, E., Chen, Y., Pedersen, J. M., Derczynski, L., & Bjerva, J. (2025) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00762 · Cited by 6
Power-up! What Can Generative Models Do for Human Computation Workflows?Preprint
Allen, G., He, G., & Gadiraju, U. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2307.02243 · Cited by 6
Ovalle, A. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2310.17533 · Cited by 6
Artificial General Intelligence (AGI) for Medical Education and TrainingPreprint
Lema, K. G. (2023) · AfricArXiv (OSF Preprints) · doi:10.31730/osf.io/xytzm · Cited by 6
A reinforcement learning approach to mitigating stereotypical biases in language modelsPreprint
Qureshi, M. R., Galárraga, L. A., & Couceiro, M. (2023) · HAL · Cited by 6
Gender Bias of LLM in Economics: An Existentialism PerspectivePreprint
Zhong, H., Chen, S., & Liang, M. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2410.19775 · Cited by 6
“In the LLM training process, the bias in the training data is mainly reflected in the following aspects(Bender et al. (2021),Abid et al. (2021),Blodgett et al. (2020),Sheng et al. (2019)): 1.”
Hussain, N., Qasim, A., Mehak, G., Zain, M., Hafeez, M., & Sidorov, G. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2510.03683 · Cited by 6
“Models such as GPT, OPT and Falcon have shown better performance in understanding nuanced language cues [19, 20].”
Who's Asking? Evaluating LLM Robustness to Inquiry Personas in Factual Question AnsweringPreprint
Akpinar, N.-J., Lee, C., Murdock, V., & Perona, P. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2510.12925 · Cited by 6
“LLMs are prone to re-create gender, race, and religious stereotypes in persona descriptions [10], occupation suggestions [35, 47], text completions [18], and other settings [1, 11, 46].”
Towards Trustworthy AI: Characterizing User-Reported Risks across LLMs "In the Wild"Preprint
Li, L., Ma, R., Xue, Z., & Xiong, J. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2509.08912 · Cited by 6
“These biases further manifest as misrepresentation, stereotyping, and derogatory language [34], such as gender biases in job market [57] and anti-Muslim bias [1].”
SESGO: Spanish Evaluation of Stereotypical Generative OutputsPreprint
Robles, M., Bernal, C., Raigoso, D., & Rubio, M. D. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2509.03329 · Cited by 6
Us-vs-Them bias in Large Language ModelsPreprint
Prama, T. T., Zimmerman, J. W., Danforth, C. M., & Dodds, P. S. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2512.13699 · Cited by 6
“Studies have shown that LLMs often reflect human-like prejudices re-lated to gender, ethnicity, and religion (Abid et al., 2021; Ahn and Oh, 2021; Bordia and Bowman, 2019), echoing earlier findings in word embeddings trained on large web corpora (Caliskan et al., 2016).”
Quantifying Fairness in LLMs Beyond Tokens: A Semantic and Statistical PerspectivePreprint
Xu, W., Wang, Y., Xue, C., Hu, X., Fang, X., Dong, G., & Reddy, C. K. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2506.19028 · Cited by 6
“…to detect and measure potential biases in LLMs. Demographic representation evaluation methods (Brown et al., 2020; Liang et al., 2022; Mattern et al., 2022; Smith et al., 2022b; Abid et al., 2021; Barikeri et al., 2021) confine the definition of bias to a narrow and commonly used vocabulary.”
Enhancing Patient-Centric Communication: Leveraging LLMs to Simulate Patient PerspectivesPreprint
Ma, X., Zhu, R., Wang, Z., Xiong, J., Chen, Q., Tang, H., Camp, L. J., & Ohno-Machado, L. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2501.06964 · Cited by 6
“These biases can manifest in their outputs, and under certain conditions, such as when persona-based prompts are used, they can be magnified [1, 8, 12].”
Controlling Large Language Model Outputs: A Primer
Ji, J., Goldstein, J., & Lohn, A. J. (2023) · doi:10.51593/2023ca009 · Cited by 6
Algorithmic bias, generalist models, and clinical medicine
Keeling, G. (2024) · AI and Ethics · doi:10.1007/s43681-023-00329-x · Cited by 5
“To illustrate: In one recent study, Abid et al. (2021) found that in 66/100 cases GPT-3 completed the prompt ‘two Muslims walked into a’ with violent phraseology including references to killing and shooting.”
Case 429: A Murder Mystery Game for Teaching Bias in Generative AI
Maheshwary, P., Haiman, A., Brown, E., Sangekar, A., & Wang, C. (2025) · Annual Symposium on Computer-Human Interaction in Play · doi:10.1145/3744736.3749366 · Cited by 5
“This challenge is particularly acute with large language models (LLMs), which often perpetuate harmful stereotypes (e.g., associating ”Muslim” with violence [2], or underrepresenting older adults [3]) while presenting outputs with authoritative confidence.”
Measuring stereotype and deviation biases in large language models
Wang, D., Brignac, E., Mao, M., & Fang, X. (2026) · Scientific Reports · doi:10.1038/s41598-026-52923-8 · Cited by 5
“Statistical tests confirm that these divergences from real-world political affiliation distributions are highly significant.”
From Charts to Fair Narratives: Uncovering and Mitigating Geo-Economic Biases in Chart-to-Text
Mahbub, R., Islam, M. S., Nayeem, M. T., Laskar, M. T. R., Rahman, M., Joty, S., & Hoque, E. (2025) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2025.emnlp-main.1472 · Cited by 5
Surfacing Subtle Stereotypes: A Multilingual, Debate-Oriented Evaluation of Modern LLMs
Saeed, M., Abdul-Mageed, M., & Shehata, S. (2026) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/4u9x3z4g8jfk · Cited by 5
Between search and platform: ChatGPT under the DSA
Lorente, T., & Gardhouse, K. (2026) · Internet Policy Review · doi:10.14763/2026.1.2084 · Cited by 5
MEDEQUALQA: Evaluating Biases in LLMs with Counterfactual Reasoning
Ghosh, R., Gupta, A., McBride, H., Vaidya, A. J., & Mahmood, F. (2025) · doi:10.18653/v1/2025.sciprodllm-1.4 · Cited by 5
Johnson, R. L., Duran, L. D. D., Panai, E., González, N. M., Pistilli, G., Kalpokienė, J., & Bertulfo, D. J. (2026) · AI and Ethics · doi:10.1007/s43681-026-01038-x · Cited by 6
Anchoring Bias in LLM-as-a-Judge Systems: Prior Scores Compromise Evaluation Independence
Kapetanović, A. L., Altwlkany, K., Merćep, A., Đuričić, T., & Lacić, E. (2026) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3799682.3840879 · Cited by 6
Bjørn, P., Menéndez-Blanco, M., & Borsotti, V. (2022) · Springer eBooks · doi:10.1007/978-3-031-13314-5_7 · Cited by 5
Towards Large Language Model Organization: A Case Study on Abstractive Summarization
Boros, K., & Oyamada, M. (2023) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata59044.2023.10386199 · Cited by 5
“While these novel approaches yield state-of-the-art results across various tasks, they exhibit significant shortcomings such as hallucinations, high computational costs, or gender bias [3] [4].”
BackgroundWith a Grain of SALT: Are LLMs Fair Across Social Dimensions?
Arif, S., Khan, Z., Butt, M. K., Rashid, M. S., Raza, A. A., & Athar, A. (2026) · Workshop on Trustworthy NLP (TrustNLP) · doi:10.18653/v1/2026.trustnlp-main.48 · Cited by 5
Sports and Women’s Sports: Gender Bias in Text Generation with Olympic Data
Biester, L. (2025) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2025.naacl-short.17 · Cited by 5
WoNBias: A Dataset for Classifying Bias & Prejudice Against Women in Bengali Text
Aupi, M. R. I., Tafannum, N., Rahman, M. S., Hassan, K. M., & Rahman, N. (2025) · Workshop on Gender Bias in Natural Language Processing (GeBNLP) · doi:10.18653/v1/2025.gebnlp-1.10 · Cited by 5
Not Quite ‘Ask a Librarian’: AI on the Nature, Value, and Future of LIS
Dinneen, J. D., & Bubinger, H. (2021) · Proceedings of the Association for Information Science and Technology · doi:10.1002/pra2.441 · Cited by 5
“Future studies could codify and compare such methods, perhaps by drawing on recent evaluations of bias in the outputs of AI language models (c.f. Abid et al. , 2021, published just after the present manuscript was accepted) and collecting data using the given system’s own API rather than going…”
BackgroundMethodRoadblocks in Gender Bias Measurement for Diachronic Corpora
Alshahrani, S., Wali, E., Alshamsan, A. R., Chen, Y., & Matthews, J. N. (2022) · Workshop on Computational Approaches to Historical Language Change (LChange) · doi:10.18653/v1/2022.lchange-1.15 · Cited by 5
“At the same time, it has been demonstrated that these systems learn a wide variety of societal biases embedded in human text including racial bias, gender bias, and religious bias (Caliskan et al., 2017; Abid et al., 2021).”
BackgroundStudent-Teacher Prompting for Red Teaming to Improve Guardrails
Llaca, R. R., Leskoschek, V., Paiva, V. C., Lupău, C., Lippmann, P., & Yang, J. (2023) · doi:10.18653/v1/2023.artofsafety-1.2 · Cited by 5
“The earliest iterations of LLM red-teaming consisted of attacks of a single type, usually representational bias (Abid et al., 2021), and used template-based adversarial data (Ribeiro et al., 2020; Xu et al., 2021).”
BackgroundSearch Engines, White Ignorance, and the Social Epistemology of Technology
Habgood‐Coote, J. (2024) · Philosophy · doi:10.1017/s003181912400007x · Cited by 5
“[OpenAI], 2020; Abid et al ., 2021; Bender et al ., 2021), and there is evidence that Chat-GPT3 avoided producing problematic outputs by employing a large underpaid workforce in Kenya to label problematic results (Perrigo, 2023).”
Generating language assessment content free from representational harms
Choi, I., & Zu, J. (2025) · Language Testing · doi:10.1177/02655322251349560 · Cited by 5
“In another example, Abid et al. (2021) asked a language model to complete the sentence fragment: “Audacious is to boldness as [religious adjective] is to.””
Efficient LLM-Jailbreaking via Multimodal-LLM Jailbreak
Ji, H., Lin, Z., Niu, Z., Gao, X., & Hua, G. (2026) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v40i42.40863 · Cited by 5
Developing trustworthy AI applications with foundation modelsPreprint
Möck, M., Schmidt, S., Müller, F. B., Görge, R., Schmitz, A., Haedecke, E., Voss, A., Hecker, D., & Poretschkin, M. (2024) · arXiv (Cornell University) · doi:10.24406/publica-2987 · Cited by 5
Developing an occupational prestige scale using Large Language ModelsPreprint
de Vries, R., Hill, M., & Ruis, L. (2024) · SocArXiv (OSF Preprints) · doi:10.31235/osf.io/ct4vz · Cited by 5
Blodgett, S. L., Curry, A. C., Dev, S., Madaio, M., Nenkova, A., Yang, D., & Xiao, Z. (2024) · doi:10.18653/v1/2024.hcinlp-1 · Cited by 5
Analyzing the Safety of Japanese Large Language Models in Stereotype-Triggering PromptsPreprint
Nakanishi, A., Sano, Y., Liu, G., & Pierri, F. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2503.01947 · Cited by 5
“While bias-reducing algorithms or implementing safety prompts has shown some improvements [39], [40], comprehensive analyses are still lacking.”
The impact of Large Language Models on the publishing sectors : Books, academic journals, newspapers
Kulesz, O. (2023) · DiVA (Linnaeus University) · Cited by 5
Cheng, Y., Moakhar, A. S., Fan, C., Hosseini, P., Faghih, K., Sodagar, Z., Wang, W., & Feizi, S. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2510.23853 · Cited by 7
“Alignment spans capabilities like instruction following (Zhou et al., 2023; Qin et al., 2024), harm avoidance (Chao et al., 2024), bias mitigation (Abid et al., 2021; Lucy and Bam-man, 2021), hallucination reduction (Tonmoy et al., 2024), and misuse prevention (Sandbrink, 2023).”
Large Language Models Develop Novel Social Biases Through Adaptive ExplorationPreprint
Wu, A. J., Liu, R., Bai, X., & Griffiths, T. L. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2511.06148 · Cited by 5
“However, all frontier LLMs produced even more stratified outcomes than humans (mean SI = 1 .”
Metadata
- Authors
- Abid, A., Farooqi, M., & Zou, J.
- Affiliations
- Stanford University (US)
- McMaster University (CA)
- Venue
- Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society
- Publisher
- ACM
- Published
- 2021-07-21
- Pages
- 298-306
- ISSN
- 3065-8365 (ISSN-L)
- OpenAlex
- W3184144760
- Cited by
- 476
- References
- 3
- Citations by year
- 2026: 81 · 2025: 124 · 2024: 105 · 2023: 97 · 2022: 36 · 2021: 33
- FWCI
- 61.36
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- Proceedings of the AAAI/ACM Conference on AI Ethics and Society (publishedVersion)
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Artificial Intelligence › Hate Speech and Cyberbullying Detection
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Keywords
- Islamophobia, religious bias, large language models, GPT-3, analogical reasoning, adversarial prompts, story generation, bias mitigation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.