Persistent Anti-Muslim Bias in Large Language Models
Open accessThe DOI matches its Crossref record; 13 of the 13 references it cites were found in a registry or an index.
Abstract
It has been observed that large-scale language models capture undesirable societal biases, e.g. relating to race and gender; yet religious bias has been relatively unexplored. We demonstrate that GPT-3, a state-of-the-art contextual language model, captures persistent Muslim-violence bias. We probe GPT-3 in various ways, including prompt completion, analogical reasoning, and story generation, to understand this anti-Muslim bias, demonstrating that it appears consistently and creatively in different uses of the model and that it is severe even compared to biases about other religious groups. For instance, Muslim is analogized to terrorist in 23% of test cases, while Jewish is mapped to its most common stereotype, money, in 5% of test cases. We quantify the positive distraction needed to overcome this bias with adversarial text prompts, and find that use of the most positive 6 adjectives reduces violent completions for Muslims from 66% to 20%, but which is still higher than for other religious groups.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Pre-trained on massive amounts of text on the Internet, LMs are known to produce linguistically fawed, factually incorrect, or even ethically problematic text at times [1,24].”
From CoAuthor: Designing a Human-AI Collaborative Writing Dataset for Exploring Language Model Capabilities (2022) · cites this paper as [1]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“LLMs and other AI tools may have a number of biases that could arise from various factors, such as unbalanced training datasets74–77.”
From On opportunities and challenges of large multimodal foundation models in education (2025) · cites this paper as [75]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“it is also a liability; all AI systems might inherit the same problematic biases of a few foundation models [Bolukbasi et al. 2016; Caliskan et al. 2017; Abid et al. 2021, inter alia]”
From On the Opportunities and Risks of Foundation Models (2021) · cites this paper as Abid et al. 2021
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Previous work has shown that language models tend to exhibit human-like biases with respect to specific protected groups such as gender, ethnicity or religious orientation3–6.”
From Generative language models exhibit social identity biases (2024) · cites this paper as [4]
In the citing paper
“If a medical LLM has been trained on data that contains misinformation or biased content, LLMs deployed for clinical decision-making are at risk of reproducing these problematic contents [120,124–126].”
From Bias in medical AI: Implications for clinical decision-making (2024) · cites this paper as [126]
1 more claim about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404 · cited 1×
“These languagemodels have evolved, from learning individual word vectors with single-layer models [8], to more complex language generation architectures such as recurrent neural networks [5] and most recently transformers [4, 6, 14].”
BackgroundEfficient Estimation of Word Representations in Vector Space
Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1301.3781 · Cited by 26,568 · cited 1×
“These languagemodels have evolved, from learning individual word vectors with single-layer models [8], to more complex language generation architectures such as recurrent neural networks [5] and most recently transformers [4, 6, 14].”
BackgroundLanguage Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372 · cited 4×
“These languagemodels have evolved, from learning individual word vectors with single-layer models [8], to more complex language generation architectures such as recurrent neural networks [5] and most recently transformers [4, 6, 14].”
BackgroundMethodInfluentialSelf-Critical Sequence Training for Image Captioning
Rennie, S. J., Marcheret, E., Mroueh, Y., Ross, J., & Goel, V. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.131 · Cited by 2,112 · cited 1×
“These descriptions are generated by passing images through a pretrained image captioning model [11], and fine-tuning GPT-3 to transform the short captions into long descriptions, as shown in Fig.”
MethodReformer: The Efficient Transformer
Kitaev, N., Kaiser, Ł., & Levskaya, A. (2020) · arXiv · doi:10.48550/arxiv.2001.04451 · Cited by 1,435 · cited 1×
“These languagemodels have evolved, from learning individual word vectors with single-layer models [8], to more complex language generation architectures such as recurrent neural networks [5] and most recently transformers [4, 6, 14].”
BackgroundSemi-supervised Sequence Learning
Dai, A. M., & Le, Q. V. (2015) · arXiv · doi:10.48550/arxiv.1511.01432 · Cited by 994
StereoSet: Measuring stereotypical bias in pretrained language models
Nadeem, M., Bethke, A., & Reddy, S. (2020) · arXiv · doi:10.48550/arxiv.2004.09456 · Cited by 643 · cited 2×
“As a result, even though the various language models have different architectures, since they are trained on similar corpora of text, they often learn similar biases [9].”
BackgroundUniversal Adversarial Triggers for Attacking and Analyzing NLP
Wallace, E., Feng, S., Kandpal, N., Gardner, M., & Singh, S. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1221 · Cited by 642
Identifying and Reducing Gender Bias in Word-Level Language Models
Bordia, S., & Bowman, S. R. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Student Research Workshop · doi:10.18653/v1/n19-3002 · Cited by 219 · cited 1×
“Previous work has explored the biases that are ultimately learned by language models, with the primary focus being racial and gender stereotypes [2, 3, 7, 12].”
BackgroundThe Woman Worked as a Babysitter: On Biases in Language GenerationPreprint
Sheng, E., Chang, K.-W., Natarajan, P., & Peng, N. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.01326 · Cited by 205 · cited 2×
“In [12], the authors made similar observations about language generation models.”
Background
Research works printed but not linked: 6
- 7. T. Bolukbasi, K.-W. Chang, J. Y. Zou, V. Saligrama, A. T. Kalai, Advances in neural information processing systems 29 , 4349 (2016).
- 9. S. Bordia, S. R. Bowman, arXiv preprint arXiv:1904.03035 (2019). · arXiv
- 2. A. M. Dai, Q. V. Le, Advances in neural information processing systems 28 , 3079 (2015).
- 10. K. Lu, P. Mardziel, F. Wu, P. Amancharla, A. Datta, Logic, Language, and Security (Springer, 2020), pp. 189-202.
- 3. A. Vaswani, et al. , Advances in neural information processing systems (2017), pp. 59986008.
- 14. E. Wallace, S. Feng, N. Kandpal, M. Gardner, S. Singh, arXiv preprint arXiv:1908.07125 (2019). · arXiv
Cited by
On the Opportunities and Risks of Foundation Models
Bommasani, R., Hudson, D. A., Adeli, E., Altman, R., Arora, S., von Arx, S., Bernstein, M. S., Bohg, J., Bosselut, A., Brunskill, E., et al. (2021) · arXiv · doi:10.48550/arxiv.2108.07258 · Cited by 2,676
PaLM: Scaling Language Modeling with Pathways
Chowdhery, A., Narang, S., Devlin, J., Bosma, M., Mishra, G., Roberts, A., Barham, P., Chung, H. W., Sutton, C., Gehrmann, S., et al. (2022) · arXiv · doi:10.48550/arxiv.2204.02311 · Cited by 2,590
“In particular we see in Table 37 that Islam is associated with highly charged terms and stereotypes such as terrorist, violent and radical, similar to other evaluations of decoder-only models revealing anti-Muslim bias (Abid et al., 2021).”
ResultGPTs are GPTs: An Early Look at the Labor Market Impact Potential of Large Language Models
Eloundou, T., Manning, S., Mishkin, P., & Rock, D. (2023) · arXiv · doi:10.48550/arxiv.2303.10130 · Cited by 2,330
“…partly because out-of-the-box general-purpose LLMs may continue to be unreliable for various tasks due to issues such as factual inaccuracies, inherent biases, privacy concerns, and disinformation risks (Abid et al., 2021; Schramowski et al., 2022; Goldstein et al., 2023; OpenAI, 2023a).”
BackgroundA Survey of Large Language Models
Zhao, W. X., Zhou, K., Li, J., Tang, T., Dong, Z., Hou, Y., Zhang, B., Min, Y., Zhang, J., Liu, P., et al. (2026) · Frontiers of Computer Science · doi:10.1007/s11704-026-60308-3 · Cited by 2,093
Evaluating Large Language Models Trained on Code
Chen, M., Tworek, J., Jun, H., Yuan, Q., Pinto, H. P. D. O., Kaplan, J., Edwards, H., Burda, Y., Joseph, N., Brockman, G., et al. (2021) · arXiv · doi:10.48550/arxiv.2107.03374 · Cited by 1,436
“Mirroring what has been found in the case of other language models trained on Internet data (Bender et al., 2021; Blod-gett et al., 2020; Abid et al., 2021; Brown et al., 2020”
BackgroundBias and Fairness in Large Language Models: A Survey
Gallegos, I. O., Rossi, R. A., Barrow, J., Tanjim, M. M., Kim, S., Dernoncourt, F., Yu, T., Zhang, R., & Ahmed, N. K. (2024) · Computational Linguistics · doi:10.1162/coli_a_00524 · Cited by 1,245
“In the first example of modified prompting language, positive triggers are added to the input to condition the model to generate more positive outputs (based on Abid et al. (2021) and Narayanan Venkit et al.”
BackgroundMethodInfluentialTree of Attacks: Jailbreaking Black-Box LLMs Automatically
Mehrotra, A., Zampetakis, M., Kassianik, P., Nelson, B. A., Anderson, H. S., Singer, Y., & Karbasi, A. (2024) · neural information processing systems · doi:10.52202/079017-1952 · Cited by 1,085
Can Large Language Models Transform Computational Social Science?
Ziems, C., Held, W., Shaikh, O., Chen, J., Zhang, Z., & Yang, D. (2023) · arXiv · doi:10.48550/arxiv.2305.03514 · Cited by 839
“Most notably, LLMs are known to amplify social biases and stereotypes (Sheng et al., 2021; Abid et al., 2021; Borchers et al., 2022; Lucy and Bamman, 2021; Shaikh et al., 2022).”
BackgroundAnil, R., Dai, A. M., Firat, O., Johnson, M., Lepikhin, D., Passos, A., Shakeri, S., Taropa, E., Bailey, P., Chen, Z., et al. (2023) · arXiv · doi:10.48550/arxiv.2305.10403 · Cited by 834
LaMDA: Language Models for Dialog Applications
Thoppilan, R., De Freitas, D., Hall, J., Shazeer, N., Kulshreshtha, A., Cheng, H.-T., Jin, A., Bos, T., Baker, L., Du, Y., et al. (2022) · arXiv · doi:10.48550/arxiv.2201.08239 · Cited by 778
Taxonomy of Risks posed by Language Models
Weidinger, L., Uesato, J., Rauh, M., Griffin, C., Huang, P.-S., Mellor, J., Glaese, A., Cheng, M., Balle, B., Kasirzadeh, A., et al. (2022) · 2022 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3531146.3533088 · Cited by 764
“in 23% of test cases [2]; and gender bias, presenting fictional female characters as more domestic than male counterparts [120].”
BackgroundBeyond the Imitation Game: Quantifying and extrapolating the capabilities of language models
Srivastava, A., Rastogi, A., Rao, A., A., M. S. A., Abid, A., Fisch, A., Brown, A. R., Santoro, A., Gupta, A., Garriga-Alonso, A., et al. (2023) · BOA (University of Milano-Bicocca) · Cited by 737
Self-citationBias in medical AI: Implications for clinical decision-making
Cross, J. L., Choma, M. A., & Onofrey, J. A. (2024) · PLOS Digital Health · doi:10.1371/journal.pdig.0000651 · Cited by 607
“If a medical LLM has been trained on data that contains misinfor-mation or biased content, LLMs deployed for clinical decision-making are at risk of reproducing these problematic contents [120,124–126].”
MethodCan Large Language Models Transform Computational Social Science?
Ziems, C., Held, W., Shaikh, O., Chen, J., Zhang, Z., & Yang, D. (2024) · Computational Linguistics · doi:10.1162/coli_a_00502 · Cited by 585
“Most notably, LLMs are known to amplify social biases and stereotypes (Sheng et al., 2021; Abid et al., 2021; Borchers et al., 2022; Lucy and Bamman, 2021; Shaikh et al., 2022).”
BackgroundBeyond the Imitation Game: Quantifying and extrapolating the capabilities of language models
Srivastava, A., Rastogi, A., Rao, A., Shoeb, A. A. M., Abid, A., Fisch, A., Brown, A. R., Santoro, A., Gupta, A., Garriga-Alonso, A., et al. (2022) · arXiv · doi:10.48550/arxiv.2206.04615 · Cited by 569
Self-citationA Categorical Archive of ChatGPT Failures
Borji, A. (2023) · doi:10.21203/rs.3.rs-2895792/v1 · Cited by 514
A Categorical Archive of ChatGPT Failures
Borji, A. (2023) · arXiv · doi:10.48550/arxiv.2302.03494 · Cited by 514
Ethical and social risks of harm from Language Models
Weidinger, L., Mellor, J., Rauh, M., Griffin, C., Uesato, J., Huang, P.-S., Cheng, M., Glaese, M., Balle, B., Kasirzadeh, A., et al. (2021) · arXiv · doi:10.48550/arxiv.2112.04359 · Cited by 477
Ghafouri, V., Agarwal, V., Zhang, Y., Sastry, N. R., Such, J., & Suárez‐Tangil, G. (2023) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3583780.3614777 · Cited by 484
“For instance, Abid et al. [1] demonstrate that the GPT-3 language model carries undesirable societal biases about religious groups.”
BackgroundLarge Language Model Alignment: A Survey
Shen, T., Jin, R., Huang, Y., Liu, C., Dong, W., Guo, Z., Wu, X., Liu, Y., & Xiong, D. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2309.15025 · Cited by 36
“, 2020) has been shown to exhibit religious bias (Abid et al., 2021) and gender bias (Lucy and Bamman, 2021) when freely generating stories.”
BackgroundExplicitly unbiased large language models still form biased associations
Bai, X., Wang, A., Sucholutsky, I., & Griffiths, T. L. (2025) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2416228122 · Cited by 442
Gender bias and stereotypes in Large Language Models
Kotek, H., Dockum, R., & Sun, D. (2023) · Proceedings of The ACM Collective Intelligence Conference · doi:10.1145/3582269.3615599 · Cited by 381
Investigating Cultural Alignment of Large Language Models
AlKhamissi, B., ElNokrashy, M., Alkhamissi, M., & Diab, M. (2024) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2024.acl-long.671 · Cited by 379
Biases in Large Language Models: Origins, Inventory, and Discussion
Navigli, R., Conia, S., & Ross, B. (2023) · Journal of Data and Information Quality · doi:10.1145/3597307 · Cited by 364
“Nevertheless, large language models have been found to exhibit religion bias consistently in different tasks and uses [1].”
BackgroundResultOpportunities and Challenges for ChatGPT and Large Language Models in Biomedicine and Health
Tian, S., Jin, Q., Yeganova, L., Lai, P.-T., Zhu, Q., Chen, X., Yang, Y., Chen, Q., Kim, W., Comeau, D. C., et al. (2023) · Briefings in Bioinformatics · doi:10.1093/bib/bbad493 · Cited by 359
“In a recent study, researchers show that text generated with GPT-3 can capture social bias [114].”
BackgroundGuiding Pretraining in Reinforcement Learning with Large Language ModelsPreprint
Du, Y., Watkins, O., Wang, Z., Colas, C., Darrell, T., Abbeel, P., Gupta, A. K., & Andreas, J. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2302.06692 · Cited by 347
“While LLMs priors have been shown to exhibit impressive common-sense capabilities, it is also well-known that such models are highly prone to harmful social biases and stereotypes (Bender et al., 2021; Abid et al., 2021; Nadeem et al., 2020).”
BackgroundLee, M., Liang, P., & Yang, Q. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3502030 · Cited by 345
“Pre-trained on massive amounts of text on the Internet, LMs are known to produce linguistically flawed, factually incorrect, or even ethically problematic text at times [1, 24].”
BackgroundLook Before You Leap: An Exploratory Study of Uncertainty Analysis for Large Language Models
Huang, Y., Song, J., Wang, Z., Zhao, S., Chen, H., Juefei-Xu, F., & Ma, L. (2025) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2024.3519464 · Cited by 339
“Such phenomenon of erroneous generation can exhibit in terms of different manifestations (e.g., hallucination [8], disinformation [9], bias [10]) across various tasks.”
Bias and Unfairness in Information Retrieval Systems: New Challenges in the LLM Era
Dai, S., Chen, X., Xu, S., Pang, L., Dong, Z., & Xu, J. (2024) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3637528.3671458 · Cited by 331
“For example, when training LLMs, it is common to encounter discriminatory content [3, 29, 38].”
BackgroundIn-Context Impersonation Reveals Large Language Models' Strengths and Biases
Salewski, L., Alaniz, S., Rio-Torto, I., Schulz, E., & Akata, Z. (2023) · neural information processing systems · doi:10.52202/075280-3152 · Cited by 311
AI Suggestions Homogenize Writing Toward Western Styles and Diminish Cultural Nuances
Agarwal, D., Naaman, M., & Vashistha, A. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713564 · Cited by 307
“For example, Qadri et al. [68] showed that text-to-image models fail to generate cultural artifacts, amplify hegemonic cultural defaults, and perpetuate cultural tropes when depicting non-Western cultures.”
A Survey on Fairness in Large Language ModelsPreprint
Li, Y., Du, M., Song, R., Wang, X., & Ying, W. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2308.10149 · Cited by 299
Large pre-trained language models contain human-like biases of what is right and wrong to do
Schramowski, P., Turan, C., Andersen, N., Rothkopf, C. A., & Kersting, K. (2022) · Nature Machine Intelligence · doi:10.1038/s42256-022-00458-8 · Cited by 291
“However, since it was trained on unfiltered text data from the internet, it may inherit biased and toxic knowledge, which can be indeed observed [15, 16].”
BackgroundGender and Representation Bias in GPT-3 Generated Stories
Lucy, L., & Bamman, D. (2021) · Proceedings of the Third Workshop on Narrative Understanding · doi:10.18653/v1/2021.nuse-1.5 · Cited by 287
“Abid et al. (2021) showed that GPT-3’s association of Mus-lims and violence can be difficult to diminish, even when prompts include anti-stereotype content.”
BackgroundSociotechnical Harms of Algorithmic Systems: Scoping a Taxonomy for Harm Reduction
Shelby, R., Rismani, S., Henne, K., Moon, A., Rostamzadeh, N., Nicholas, P., Yilla-Akbari, N., Gallegos, J., Smart, A., Garcia, E., et al. (2023) · Proceedings of the 2023 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3600211.3604673 · Cited by 279
“Similarly, patterns of demeaning imagery have been found in hateful natural language predictions about Muslim people [1], and toxicity and sentiment classifiers that are more likely to classify descriptions or mentions of disabilities [113, 213] and LGBTQ identities [215, 237] as toxic or negative.”
BackgroundMental-LLM: Leveraging Large Language Models for Mental Health Prediction via Online Text Data
Xu, X., Yao, B., Dong, Y., Gabriel, S., Yu, H., Hendler, J., Ghassemi, M., Dey, A. K., & Wang, D. (2024) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3643540 · Cited by 278
TrustLLM: Trustworthiness in Large Language Models
Huang, Y., Sun, L., Wang, H., Wu, S., Zhang, Q., Li, Y., Gao, C., Huang, Y., Lyu, W., Zhang, Y., et al. (2024) · arXiv · doi:10.48550/arxiv.2401.05561 · Cited by 256
Self-citationHolistic Evaluation of Language Models
Liang, P., Bommasani, R., Lee, T., Tsipras, D., Soylu, D., Yasunaga, M., Zhang, Y., Narayanan, D., Wu, Y., Kumar, A., et al. (2022) · arXiv · doi:10.48550/arxiv.2211.09110 · Cited by 255
Bias in Large Language Models: Origin, Evaluation, and Mitigation
Guo, Y., Guo, M., Su, J., Yang, Z., Zhu, M., Li, H., Qiu, M., & Liu, S. S. (2026) · Electronics · doi:10.3390/electronics15091824 · Cited by 249
“Abid et al. (2021) demonstrate that LLMs captures Muslim-violence bias in various uses cases and the Muslin-violence bias is more severe than commonly-seen biases of other religious groups.”
Jailbreaking Attack against Multimodal Large Language ModelPreprint
Niu, Z., Ren, H., Gao, X., Hua, G., & Jin, R. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2402.02309 · Cited by 244
“These models exhibit advanced general abilities but also pose serious safety risks such as truthfulness, toxicity, and bias (Gehman et al., 2020; Perez et al., 2022; Sheng et al., 2019; Abid et al., 2021; Carlini et al., 2021).”
BackgroundBeyond the Imitation Game: Quantifying and extrapolating the capabilities of language models
Srivastava, A., Rastogi, A., Rao, A., Shoeb, A. A. M., Abid, A., Fisch, A., Brown, A. R., Santoro, A., Gupta, A., Garriga-Alonso, A., et al. (2022) · UCL Discovery (University College London) · Cited by 241
Self-citationAI generates covertly racist decisions about people based on their dialect
Hofmann, V., Kalluri, P. R., Jurafsky, D., & King, S. (2024) · Nature · doi:10.1038/s41586-024-07856-5 · Cited by 233
“14 and 15 and Supplementary Tables 17–19).”
InfluentialSelf-Diagnosis and Self-Debiasing: A Proposal for Reducing Corpus-Based Bias in NLP
Schick, T., Udupa, S., & Schütze, H. (2021) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00434 · Cited by 227
“…et al., 2016; Caliskan et al., 2017; Zhao et al., 2017; Rudinger et al., 2018; Gonen and Goldberg, 2019; Bordia and Bowman, 2019; Sheng et al., 2019; Basta et al., 2019; Nangia et al., 2020, i.a. ) and are prone to generating toxic texts (Brown et al., 2020; Gehman et al., 2020; Abid et al., 2021).”
BackgroundEthical ChatGPT: Concerns, Challenges, and Commandments
Zhou, J., Müller, H., Holzinger, A., & Chen, F. (2024) · Electronics · doi:10.3390/electronics13173417 · Cited by 128
Safe Latent Diffusion: Mitigating Inappropriate Degeneration in Diffusion Models
Schramowski, P., Brack, M., Deiseroth, B., & Kersting, K. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.02157 · Cited by 219
“[1] Abubakar Abid, Maheen Farooqi, and James Zou.”
BackgroundExplanations from Large Language Models Make Small Reasoners Better
Li, S., Chen, J., Shen, Y., Chen, Z., Zhang, X., Li, Z., Wang, H., Jing, Q., Peng, B., Mao, Y., et al. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2210.06726 · Cited by 32
“Our work is built on top of explanations generated from LLM, which have been observed to capture gender, race and religion biases (Brown et al., 2020; Lucy and Bamman, 2021; Abid et al., 2021).”
MethodAI Transparency in the Age of LLMs: A Human-Centered Research Roadmap
Liao, Q. V., & Wortman Vaughan, J. (2024) · Harvard Data Science Review · doi:10.1162/99608f92.8036d03b · Cited by 210
Bias in Large Language Models: Origin, Evaluation, and Mitigation
Guo, Y., Guo, M., Su, J., Yang, Z., Zhu, M., Li, H., Qiu, M., & Liu, S. S. (2024) · arXiv · doi:10.48550/arxiv.2411.10915 · Cited by 208
“Abid et al. (2021) demonstrate that LLMs captures Muslim-violence bias in various uses cases and the Muslin-violence bias is more severe than commonly-seen biases of other religious groups.”
Ghosh, S., & Caliskan, A. (2023) · Proceedings of the 2023 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3600211.3604672 · Cited by 112
A Bibliometric Review of Large Language Models Research from 2017 to 2023
Fan, L., Li, L., Ma, Z., Lee, S., Yu, H., & Hemphill, L. (2024) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3664930 · Cited by 196
“For example, there is persistent anti-Muslim bias in some LLMs (Abid et al., 2021).”
Foundations & Trends in Multimodal Machine Learning: Principles, Challenges, and Open Questions
Liang, P. P., Zadeh, A., & Morency, L.-P. (2024) · ACM Computing Surveys · doi:10.1145/3656580 · Cited by 195
“Finally, one of the biggest challenges facing multimodal generation is difficulty in evaluating generated content, especially when there exist serious ethical issues when fake news [29], hate speech [2, 79], deepfakes [89], and lip-syncing videos [245] can be easily generated.”
BackgroundTrustworthy Recommender Systems
Wang, S., Zhang, X., Wang, Y., & Ricci⋆, F. (2023) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3627826 · Cited by 195
“Technical layer”
QuRating: Selecting High-Quality Data for Training Language ModelsPreprint
Wettig, A., Gupta, A., Malik, S., & Chen, D. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2402.09739 · Cited by 199
The great Transformer: Examining the role of large language models in the political economy of AI
Luitse, D., & Denkena, W. (2021) · Big Data & Society · doi:10.1177/20539517211047734 · Cited by 190
“…scholars have raised concerns over the output of LLMs, and their potential to further soli-dify fi lter bubbles (Floridi and Chiriatti, 2020), discriminate against marginalized groups (Abid et al., 2021), and generate fake news or violence-inciting social media posts (McGuf fi e and Newhouse, 2020).”
BackgroundFoundation and large language models: fundamentals, challenges, opportunities, and social impacts
Myers, D., Mohawesh, R., Chellaboina, V. I., Sathvik, A. L., Venkatesh, P., Ho, Y.-H., Henshaw, H., Alhawawreh, M., Berdik, D., & Jararweh, Y. (2024) · Cluster Computing · doi:10.1007/s10586-023-04203-7 · Cited by 185
“Some of the results of this study were that GPT-3 associated the word ‘‘Muslim’’ with violence 66% of the time, and the term ‘‘Jewish’’ was associated with money in 5% of test cases [4].”
Tree of Attacks: Jailbreaking Black-Box LLMs Automatically
Mehrotra, A., Zampetakis, M., Kassianik, P., Nelson, B., Anderson, H., Singer, Y., & Karbasi, A. (2023) · arXiv · doi:10.48550/arxiv.2312.02119 · Cited by 183
“These templates can be based on several high-level strategies (including persona modulation [SFPT+23] and existing prompt injection techniques from cybersecurity [KLSG+24]) and can further be optimized via discrete optimization methods [ACF24].”
BackgroundLarge Language Models for Data Annotation and Synthesis: A Survey
Tan, Z., Li, D., Wang, S., Beigi, A., Jiang, B., Bhattacharjee, A., Karami, M., Li, J., Cheng, L., & Liu, H. (2024) · Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2024.emnlp-main.54 · Cited by 178
AI-Augmented Surveys: Leveraging Large Language Models and Surveys for Opinion PredictionPreprint
Kim, J., & Lee, B. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2305.09620 · Cited by 190
“Simultaneously, recent approaches utilizing the remarkable capabilities of LLMs in mimicking human attitudes and biases [14, 21, 22] have limitations in representing populations across various socio-demographic groups [21, 35], accounting for individual heterogeneity [14, 36, 37], and estimating past opinions due to t…”
BackgroundUnveiling the Implicit Toxicity in Large Language Models
Wen, J., Ke, P., Sun, H., Zhang, Z., Li, C., Bai, J., & Huang, M. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.84 · Cited by 176
You reap what you sow: On the Challenges of Bias Evaluation Under Multilingual Settings
Talat, Z., Névéol, A., Biderman, S., Clinciu, M., Dey, M., Longpre, S., Luccioni, S., Masoud, M., Mitchell, M., Radev, D., et al. (2022) · Proceedings of BigScience Episode #5 -- Workshop on Challenges & Perspectives in Creating Large Language Models · doi:10.18653/v1/2022.bigscience-1.3 · Cited by 59
“Although word embeddings may help identify biases in the context of LLMs, it is often difficult to access the learned contextual language representations of the model (Abid et al., 2021; Dev et al., 2020).”
BackgroundNationality Bias in Text Generation
Narayanan Venkit, P., Gautam, S., Panchanadikar, R., Huang, T.-H., & Wilson, S. (2023) · Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2023.eacl-main.9 · Cited by 39
Kim, B., Kim, H., Lee, S.-W., Lee, G., Kwak, D., Dong Hyeon, J., Park, S., Kim, S., Kim, S., Seo, D., et al. (2021) · Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2021.emnlp-main.274 · Cited by 72
“Many studies have already reported that these biases from training data have significant influences on large-scale language models as well (Abid et al., 2021; Garrido-Muñoz et al., 2021; Shwartz and Choi, 2020).”
BackgroundKlein, L. F., & D’Ignazio, C. (2024) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3630106.3658543 · Cited by 173
“…are reinforcing binaries – gender binaries, patriarchal hierarchies, and paternalistic stereotypes – and if so, instead think otherwise about how to model gender, how to build tools for gender safety, and how to create a world where technologies cease to erase and harm trans and non-binary people.”
BackgroundRed teaming ChatGPT via Jailbreaking: Bias, Robustness, Reliability and Toxicity
Zhuo, T. Y., Huang, Y., Chen, C., & Xing, Z. (2023) · arXiv · doi:10.48550/arxiv.2301.12867 · Cited by 164
“Several recent studies have demonstrated that LLMs, such as GPT-3, have a persistent bias against genders [22] and religions [23].”
BackgroundEmerging trends: A gentle introduction to fine-tuning
Church, K., Chen, Z., & Ma, Y. (2021) · Natural Language Engineering · doi:10.1017/s1351324921000322 · Cited by 165
“There are concerns about bias (Abid, Farooqi, and Zou 2021).”
Fairness in Large Language Models: A Taxonomic Survey
Chu, Z., Wang, Z., & Zhang, W. (2024) · ACM SIGKDD Explorations Newsletter · doi:10.1145/3682112.3682117 · Cited by 59
“Consequently, they could lead to discrimination against certain populations, especially in socially sensitive applications, across various dimensions such as race [5], age [43], gender [72], nation-ality [139], occupation [71], and religion [1].”
BackgroundAI auditing: The Broken Bus on the Road to AI Accountability
Birhane, A., Steed, R., Ojewale, V., Vecchione, B., & Raji, I. D. (2024) · 2024 IEEE Conference on Secure and Trustworthy Machine Learning (SaTML) · doi:10.1109/satml59370.2024.00037 · Cited by 158
“…products (e.g., YouTube comment moderation [57]); others targeted only the AI models behind the products (e.g., an investigation of tax audit models used by the IRS [58]); still others targeted only the algorithms used to build the model (e.g., pre-trained large language models such as GPT-3 [59]).”
MethodThe Illusion of Artificial Inclusion
Agnew, W., Bergman, A. S., Chien, J., Díaz, M., El-Sayed, S., Pittman, J., Mohamed, S., & McKee, K. R. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642703 · Cited by 58
“Indeed, current LLMs exhibit substantial cultural bias [14, 49, 94] and routinely produce stereotyped images of minority individuals [ 1, 104].”
BackgroundFLUTE: Figurative language understanding through textual explanations
Chakrabarty, T., Saakyan, A., Ghosh, D., & Muresan, S. (2022) · Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2022.emnlp-main.481 · Cited by 24
“, 2021), religious stereotypes (Abid et al., 2021), and gender stereotypes (Borchers et al.”
BackgroundAutomatically Auditing Large Language Models via Discrete OptimizationPreprint
Jones, E., Dragan, A. D., Raghunathan, A., & Steinhardt, J. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2303.04381 · Cited by 157
“However, LLMs can unexpectedly produce undesired behaviors; they generate toxic outputs [Gehman et al., 2020, Perez et al., 2022], exacerbate stereotypes [Sheng et al., 2019, Abid et al., 2021], and reveal private information [Carlini et al., 2020].”
BackgroundChan, A. (2023) · AI and Ethics · doi:10.1007/s43681-022-00148-6 · Cited by 155
“The authors gave GPT-3 the neutral phrase “Two Muslims walked into a…”, finding that 66 out of 100 times the completions were violence-related (involving phrases such as “shooting” or “killing”) (Abid et al. [1], p. 1).”
BackgroundSecurity and Privacy on Generative Data in AIGC: A Survey
Wang, T., Zhang, Y., Qi, S., Zhao, R., Xia, Z., & Weng, J. (2024) · ACM Computing Surveys · doi:10.1145/3703626 · Cited by 156
“The investigations revealed that GPT-3 consistently and strongly exhibits biased views against the Muslim community [8].”
BackgroundMarkl, N. (2022) · 2022 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3531146.3533117 · Cited by 78
“For example, the islamophobic predictions by GPT-3 documented in [1] are both harmful to any Muslim users who see the offensive output and feed into (and, of course, result from) much larger islamphobic discourses harming all Muslims.”
BackgroundFairness in AI-Driven Recruitment: Challenges, Metrics, Methods, and Future DirectionsPreprint
Mujtaba, D. F., & Mahapatra, N. R. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2405.19699 · Cited by 158
“Later, GPT-3 was found to generate strong negative stereotypes against religious groups [34], and other LLMs like GPT-4 and BERT have also come under scrutiny for Fig.”
BackgroundGPT3Mix: Leveraging Large-scale Language Models for Text Augmentation
Yoo, K. M., Park, D., Kang, J., Lee, S.-W., & Park, W. (2021) · Findings of the Association for Computational Linguistics: EMNLP 2021 · doi:10.18653/v1/2021.findings-emnlp.192 · Cited by 151
Arabic natural language processing for Qur’anic research: a systematic review
Bashir, M. H., Azmi, A. M., Nawaz, H., Zaghouani, W., Diab, M., Al‐Fuqaha, A., & Qadir, J. (2022) · Artificial Intelligence Review · doi:10.1007/s10462-022-10313-2 · Cited by 150
“For instance, Abid et al. (2021) have analyzed the state-of-the-art contextual language model GPT-3 and have shown that it gives persistent bias in tasks such as prompt completion, analogical reasoning,”
BackgroundDataInf: Efficiently Estimating Data Influence in LoRA-tuned LLMs and Diffusion Models
Kwon, Y., Wu, E., Wu, K., & Zou, J. (2023) · International Conference on Learning Representations · doi:10.48550/arxiv.2310.00902 · Cited by 150
“Nevertheless, even state-of-the-art models do generate factually incorrect predictions or even biased outputs [4–6], often as a result of issues in the training data.”
BackgroundSelf-citationHarms from Increasingly Agentic Algorithmic Systems
Chan, A., Salganik, R., Markelius, A., Pang, C., Rajkumar, N., Krasheninnikov, D., Langosco, L., He, Z., Duan, Y., Carroll, M., et al. (2023) · 2023 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3593013.3594033 · Cited by 149
“…and Gebru 2018; Ehsan et al. 2022; Kasy and Abebe 2021; steven t. piantadosi [@spiantado] 2022; Wolfe and Caliskan 2022], the generation of toxic language [Abid et al. 2021; Gehman et al. 2020], and informational harms [Carroll et al. 2022; Jiang et al. 2019; Li et al. 2022; Weidinger et al. 2022].”
BackgroundSynthetic data in biomedicine via generative artificial intelligence
van Breugel, B., Liu, T., Oglić, D., & van der Schaar, M. (2024) · Nature Reviews Bioengineering · doi:10.1038/s44222-024-00245-7 · Cited by 151
Yao, D., Zhang, J., Harris, I. G., & Carlsson, M. (2024) · ICASSP 2024 - 2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp48485.2024.10448041 · Cited by 43
Graph Chain-of-Thought: Augmenting Large Language Models by Reasoning on Graphs
Jin, B., Xie, C., Zhang, J., Roy, K. K., Zhang, Y., Zheng, L., Li, R., Tang, X., Wang, S., Yu, M., et al. (2024) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2024.findings-acl.11 · Cited by 148
MoralBench: Moral Evaluation of LLMs
Ji, J., Chen, Y., Jin, M., Xu, W., Hua, W., & Zhang, Y. (2025) · ACM SIGKDD Explorations Newsletter · doi:10.1145/3748239.3748246 · Cited by 150
“…with LLMs stemming from the voluminous, yet noisy, training datasets; these can lead to the inadvertent generation of biased or harmful content, such as gender and religious prejudices as well as aggressive language [52; 53; 54; 55; 56], thereby undermining their reliability and trustworthiness.”
Stereotyping Norwegian Salmon: An Inventory of Pitfalls in Fairness Benchmark Datasets
Blodgett, S. L., Lopez, G., Olteanu, A., Sim, R., & Wallach, H. (2021) · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.18653/v1/2021.acl-long.81 · Cited by 147
“…due in no small part to the deeply contextual and open nature of language use and tasks (Austin, 1975; Clark, 1996; Howcroft et al., 2020; Abid et al., 2021; Olteanu et al., 2020), and a lack of consensus about how to conceptualize or operationalize such harms (Blod-gett et al., 2020;…”
BackgroundA Short Survey of Viewing Large Language Models in Legal AspectPreprint
Sun, Z. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2303.09136 · Cited by 148
“0.2 Applications of Large Language Models in Legal Tasks Large language mode (LLMs), such as ChatGPT, have demonstrated considerable potential in various legal tasks, showcasing their ability to transform the legal domain.”
BackgroundMethodInfluentialAssessing political bias in large language models
Rettenberger, L., Reischl, M., & Schutera, M. (2025) · Journal of Computational Social Science · doi:10.1007/s42001-025-00376-w · Cited by 145
“Grounded in the theoretical framework of performative prediction [7], [8], which under-lines the active role of predictive mechanisms in shaping societal dynamics, our study employs a questionnaire-based (”Wahl-O-Mat” [9]) methodology to examine the alignment of LLMs with the views and positions of…”
When Human-AI Interactions Become Parasocial: Agency and Anthropomorphism in Affective Design
Maeda, T., & Quan-Haase, A. (2024) · The 2024 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3630106.3658956 · Cited by 144
“Training datasets and language models are not immune to biases [74], and their deployment can reinforce social stereotypes [3].”
BackgroundPerspectives on Digital Humanism
Werthner, H., Prem, E., Lee, E. A., & Ghezzi, C. (2021) · Springer eBooks · doi:10.1007/978-3-030-86144-5 · Cited by 146
Slobodkin, A., Goldman, O., Caciularu, A., Dagan, I., & Ravfogel, S. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.220 · Cited by 145
A Design Space for Intelligent and Interactive Writing Assistants
Lee, M., Gero, K. I., Chung, J. J. Y., Shum, S. B., Raheja, V., Shen, H., Venugopalan, S., Wambsganss, T., Zhou, D., Alghamdi, E. A., et al. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642697 · Cited by 136
“System is meant to evoke a sense of another agent acting on the interface or data Tool System is presented as tools, where there is not a sense of interacting with another agent Hybrid System draws from both “agent” and “tool” metaphors”
Zheng, O., Abdel-Aty, M. A., Wang, D., Wang, Z., & Ding, S. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2303.05382 · Cited by 136
“In NLP, there are some important problems owing to inductive bias from textual data, such as gender-bias [44] and racism [45].”
BackgroundWhose Language Counts as High Quality? Measuring Language Ideologies in Text Data Selection
Gururangan, S., Card, D., Dreier, S., Gade, E., Wang, L., Wang, Z., Zettlemoyer, L., & Smith, N. A. (2022) · Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2022.emnlp-main.165 · Cited by 42
Twenty‐first century technologies and language education: Charting a path forward
Kern, R. (2024) · Modern Language Journal · doi:10.1111/modl.12924 · Cited by 134
“4 Abid et al. (2021) examined GPT-3 output when the word “Muslim” was included in text-completion prompts (as in “Two Muslims walked into a…”) and found that 66% of the text completions included acts of violence.”
Generative Echo Chamber? Effect of LLM-Powered Search Systems on Diverse Information Seeking
Sharma, N., Liao, Q. V., & Xiao, Z. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642459 · Cited by 132
“Last but not least, LLMs are known to encode biases from the training data [1, 52], and can be easily steered to exhibit certain opinion biases through model adaptation techniques such as fne-tuning or prompting [6].”
BackgroundSocietal Biases in Language Generation: Progress and Challenges
Sheng, E., Chang, K.-W., Natarajan, P., & Peng, N. (2021) · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.18653/v1/2021.acl-long.330 · Cited by 131
Inducing anxiety in large language models increases exploration and bias
Coda-Forno, J., Witte, K., Jagadish, A. K., Binz, M., Akata, Z., & Schulz, E. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2304.11111 · Cited by 38
In Conversation with Artificial Intelligence: Aligning language Models with Human Values
Kasirzadeh, A., & Gabriel, I. (2023) · Philosophy & Technology · doi:10.1007/s13347-023-00606-x · Cited by 127
“Does the same notion of truth apply across a wide range of conversational contexts, or might standards of truthfulness vary according to the subject under consideration and to relevant conversational norms? Equally, there is widespread concern that the output of large-scale language models is biased [1, 15].”
BackgroundThe Security of Using Large Language Models: A Survey with Emphasis on ChatGPT
Zhou, W., Zhu, X., Han, Q.-L., Li, L., Chen, X., Wen, S., & Xiang, Y. (2025) · IEEE/CAA Journal of Automatica Sinica · doi:10.1109/jas.2024.124983 · Cited by 95
Exploring the Ethical Dimensions of Using ChatGPT in Language Learning and Beyond
Vaccino-Salvadore, S. (2023) · Languages · doi:10.3390/languages8030191 · Cited by 68
“By examining the occurrence of certain words alongside religious terms, researchers have found that words such as “violent”, “terrorism”, and “terrorist” were more commonly associated with Islam compared to other religions (Abid et al. 2021; Garrido-Muñoz et al. 2021).”
BackgroundKim, Y., Lee, J., Kim, S., Park, J., & Kim, J. (2024) · Proceedings of the 29th International Conference on Intelligent User Interfaces · doi:10.1145/3640543.3645148 · Cited by 55
MEGAnno+: A Human-LLM Collaborative Annotation System
Kim, H., Mitra, K., Chen, R. L., Rahman, S., & Zhang, D. (2024) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2024.eacl-demo.18 · Cited by 124
Metadata
- Authors
- Abid, A., Farooqi, M., & Zou, J.
- Affiliations
- Stanford University (US)
- McMaster University (CA)
- Venue
- Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society
- Publisher
- ACM
- Published
- 2021-07-21
- Pages
- 298-306
- ISSN
- 3065-8365 (ISSN-L)
- OpenAlex
- W3184144760
- Cited by
- 476
- References
- 3
- Citations by year
- 2026: 81 · 2025: 124 · 2024: 105 · 2023: 97 · 2022: 36 · 2021: 33
- FWCI
- 61.36
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- Proceedings of the AAAI/ACM Conference on AI Ethics and Society (publishedVersion)
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Artificial Intelligence › Hate Speech and Cyberbullying Detection
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Keywords
- Islamophobia, religious bias, large language models, GPT-3, analogical reasoning, adversarial prompts, story generation, bias mitigation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.