CoChat
Sign Up
PreprintCited by 3Ethics and Social Impacts of AITopic ModelingHate Speech and Cyberbullying Detection

The DOI matches its DataCite record; its reference list is not available.

Self-Debiasing Large Language Models: Zero-Shot Recognition and Reduction of Stereotypes

Open access
Gallegos, Isabel O., Rossi, Ryan A., Barrow, Joe, Tanjim, Md Mehrab, Yu, Tong, Deilamsalehy, Hanieh, Zhang, Ruiyi, Kim, Sungchul, Dernoncourt, Franck·2024·arXiv
DOI 10.48550/arxiv.2402.01981arXiv 2402.01981
Open this paper in CoChat — free accountOpen PDF

Abstract

Large language models (LLMs) have shown remarkable advances in language generation and understanding but are also prone to exhibiting harmful social biases. While recognition of these behaviors has generated an abundance of bias mitigation techniques, most require modifications to the training data, model parameters, or decoding strategy, which may be infeasible without access to a trainable model. In this work, we leverage the zero-shot capabilities of LLMs to reduce stereotyping in a technique we introduce as zero-shot self-debiasing. With two approaches, self-debiasing via explanation and self-debiasing via reprompting, we show that self-debiasing can significantly reduce the degree of stereotyping across nine different social groups while relying only on the LLM itself and a simple prompt, with explanations correctly identifying invalid assumptions and reprompting delivering the greatest reductions in bias. We hope this work opens inquiry into other zero-shot techniques for bias mitigation.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2402.01981

Cites

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Gallegos et al. (2024) proposed a prompting-based bias mitigation method called zero-shot self-debiasing.

    “Gallegos et al (2024) propose a prompting-based bias mitigation method that leverages the zero-shot capabilities of LLMs to reduce biased stereotypical predictions and call it zero-shot self-debiasing .”

    From Digital forgetting in large language models: a survey of unlearning methods (2025) · cites this paper as Gallegos et al (2024)

  • The zero-shot self-debiasing method decreases stereotyping in question-answering over nine different social groups with a single prompt.

    “The authors demonstrate the ability of their method to decrease stereotyping in questionanswering over nine different social groups with a single prompt.”

    From Digital forgetting in large language models: a survey of unlearning methods (2025) · cites this paper as Gallegos et al (2024)

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

No citation lists for this paper

No citation lists for this paper yet — checked again daily.

Cited by 3

Research works printed but not linked: 68

  1. Abubakar Abid, Maheen Farooqi, and James Zou. 2021. Persistent anti-muslim bias in large language models. In Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society, pages 298-306.
  2. Giuseppe Attanasio, Debora Nozza, Dirk Hovy, and Elena Baralis. 2022. Entropy-based attention regu- larization frees unintended bias mitigation from lists. In Findings of the Association for Computational Linguistics: ACL 2022, pages 1105-1119, Dublin, Ireland. Association for Computational Linguistics. · DOI · Paper page
  3. Emily M Bender, Timnit Gebru, Angelina McMillan- Major, and Shmargaret Shmitchell. 2021. On the dangers of stochastic parrots: Can language models be too big? In Proceedings of the 2021 ACM confer- ence on fairness, accountability, and transparency, pages 610-623.
  4. Camiel J Beukeboom and Christian Burgers. 2019. How stereotypes are shared through language: a review and introduction of the social categories and stereo- types communication (scsc) framework. Review of Communication Research, 7:1-37.
  5. Su Lin Blodgett, Solon Barocas, Hal Daumé III, and Hanna Wallach. 2020. Language (technology) is power: A critical survey of "bias" in NLP. In Pro- ceedings of the 58th Annual Meeting of the Asso- ciation for Computational Linguistics, pages 5454- 5476, Online. Association for Computational Lin- guistics. · DOI · Paper page
  6. Conrad Borchers, Dalia Gala, Benjamin Gilburt, Eduard Oravkin, Wilfried Bounsi, Yuki M Asano, and Han- nah Kirk. 2022. Looking for a handsome carpenter! debiasing GPT-3 job advertisements. In Proceedings of the 4th Workshop on Gender Bias in Natural Lan- guage Processing (GeBNLP), pages 212-224, Seattle, Washington. Association for Computational Linguis- tics. · DOI · Paper page
  7. Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners. Advances in neural information processing systems, 33:1877-1901.
  8. Lichang Chen, Jiuhai Chen, Tom Goldstein, Heng Huang, and Tianyi Zhou. 2023. Instructzero: Ef- ficient instruction optimization for black-box large language models. arXiv preprint arXiv:2306.03082. · arXiv
  9. Pengyu Cheng, Weituo Hao, Siyang Yuan, Shijing Si, and Lawrence Carin. 2021. FairFil: Contrastive neu- ral debiasing method for pretrained text encoders. In International Conference on Learning Representa- tions.
  10. Sumanth Dathathri, Andrea Madotto, Janice Lan, Jane Hung, Eric Frank, Piero Molino, Jason Yosinski, and Rosanne Liu. 2019. Plug and play language mod- els: A simple approach to controlled text generation. arXiv preprint arXiv:1912.02164. · arXiv
  11. Harnoor Dhingra, Preetiha Jayashanker, Sayali Moghe, and Emma Strubell. 2023. Queer people are peo- ple first: Deconstructing sexual identity stereo- types in large language models. arXiv preprint arXiv:2307.00101. · arXiv
  12. Emily Dinan, Angela Fan, Adina Williams, Jack Ur- banek, Douwe Kiela, and Jason Weston. 2020. Queens are powerful too: Mitigating gender bias in dialogue generation. In Proceedings of the 2020 Con- ference on Empirical Methods in Natural Language Processing (EMNLP), pages 8173-8188, Online. As- sociation for Computational Linguistics. · DOI
  13. Zahra Fatemi, Chen Xing, Wenhao Liu, and Caim- ming Xiong. 2023. Improving gender fairness of pre-trained language models without catastrophic for- getting. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Vol- ume 2: Short Papers), pages 1249-1262, Toronto, Canada. Association for Computational Linguistics.
  14. Yacine Gaci, Boualem Benattallah, Fabio Casati, and Khalid Benabdeslem. 2022. Debiasing Pretrained Text Encoders by Paying Attention to Paying At- tention. In 2022 Conference on Empirical Meth- ods in Natural Language Processing, Proceedings of the 2022 Conference on Empirical Methods in Nat- ural Language Processing, pages 9582-9602, Abu Dhabi, United Arab Emirates. Association for Com- putational Linguistics.
  15. Aparna Garimella, Akhash Amarnath, Kiran Kumar, Akash Pramod Yalla, N Anandhavelu, Niyati Chhaya, and Balaji Vasan Srinivasan. 2021. He is very intel- ligent, she is very beautiful? on mitigating social biases in language modelling and generation. In Findings of the Association for Computational Lin- guistics: ACL-IJCNLP 2021, pages 4534-4545.
  16. Aparna Garimella, Rada Mihalcea, and Akhash Amar- nath. 2022. Demographic-aware language model fine-tuning as a bias mitigation technique. In Pro- ceedings of the 2nd Conference of the Asia-Pacific Chapter of the Association for Computational Lin- guistics and the 12th International Joint Conference on Natural Language Processing, pages 311-319.
  17. Samuel Gehman, Suchin Gururangan, Maarten Sap, Yejin Choi, and Noah A. Smith. 2020. RealToxi- cityPrompts: Evaluating neural toxic degeneration in language models. In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 3356-3369, Online. Association for Computational Linguistics. · DOI · Paper page
  18. Somayeh Ghanbarzadeh, Yan Huang, Hamid Palangi, Radames Cruz Moreno, and Hamed Khanpour. 2023. Gender-tuning: Empowering fine-tuning for debi- asing pre-trained language models. In Findings of the Association for Computational Linguistics: ACL 2023, pages 5448-5458, Toronto, Canada. Associa- tion for Computational Linguistics.
  19. Yue Guo, Yi Yang, and Ahmed Abbasi. 2022. Auto- debias: Debiasing masked language models with automated biased prompts. In Proceedings of the 60th Annual Meeting of the Association for Compu- tational Linguistics (Volume 1: Long Papers), pages 1012-1023.
  20. Jacqueline He, Mengzhou Xia, Christiane Fellbaum, and Danqi Chen. 2022a. MABEL: Attenuating gen- der bias using textual entailment data. In Proceed- ings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 9681-9702, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics. · DOI
  21. Zexue He, Yu Wang, Julian McAuley, and Bod- hisattwa Prasad Majumder. 2022b. Controlling bias exposure for fair interpretable predictions. In Find- ings of the Association for Computational Linguistics: EMNLP 2022, pages 5854-5866, Abu Dhabi, United Arab Emirates. Association for Computational Lin- guistics. · DOI
  22. Alec Helbling, Mansi Phute, Matthew Hull, and Duen Horng Chau. 2023. Llm self defense: By self examination, llms know they are being tricked. arXiv preprint arXiv:2308.07308. · arXiv
  23. Jiaxin Huang, Shixiang Shane Gu, Le Hou, Yuexin Wu, Xuezhi Wang, Hongkun Yu, and Jiawei Han. 2022. Large language models can self-improve. arXiv preprint arXiv:2210.11610. · arXiv
  24. Ben Hutchinson, Vinodkumar Prabhakaran, Emily Den- ton, Kellie Webster, Yu Zhong, and Stephen Denuyl. 2020. Social biases in NLP models as barriers for persons with disabilities. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 5491-5501, Online. Association for Computational Linguistics. · DOI · Paper page
  25. Nishtha Jain, Maja Popović, Declan Groves, and Eva Vanmassenhove. 2021. Generating gender aug- mented data for NLP. In Proceedings of the 3rd Workshop on Gender Bias in Natural Language Pro- cessing, pages 93-102, Online. Association for Com- putational Linguistics. · DOI
  26. Shengyu Jia, Tao Meng, Jieyu Zhao, and Kai-Wei Chang. 2020. Mitigating gender bias amplification in distribution by posterior regularization. In Proceed- ings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 2936-2942, On- line. Association for Computational Linguistics. · DOI
  27. Masahiro Kaneko and Danushka Bollegala. 2021. De- biasing pre-trained contextualised embeddings. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Lin- guistics: Main Volume, pages 1256-1266, Online. Association for Computational Linguistics. · DOI
  28. Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yu- taka Matsuo, and Yusuke Iwasawa. 2022. Large lan- guage models are zero-shot reasoners. Advances in neural information processing systems, 35:22199- 22213.
  29. Ben Krause, Akhilesh Deepak Gotmare, Bryan McCann, Nitish Shirish Keskar, Shafiq Joty, Richard Socher, and Nazneen Fatema Rajani. 2021. GeDi: Gener- ative discriminator guided sequence generation. In Findings of the Association for Computational Lin- guistics: EMNLP 2021, pages 4929-4952, Punta Cana, Dominican Republic. Association for Compu- tational Linguistics. · DOI · Paper page
  30. Yunqi Li and Yongfeng Zhang. 2023. Fairness of chat- gpt. arXiv preprint arXiv:2305.18569. · arXiv
  31. Alisa Liu, Maarten Sap, Ximing Lu, Swabha Swayamdipta, Chandra Bhagavatula, Noah A. Smith, and Yejin Choi. 2021. DExperts: Decoding-time con- trolled text generation with experts and anti-experts. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Lan- guage Processing (Volume 1: Long Papers), pages 6691-6706, Online. Association for Computational Linguistics. · DOI
  32. Haochen Liu, Jamell Dacon, Wenqi Fan, Hui Liu, Zitao Liu, and Jiliang Tang. 2020. Does gender matter? towards fairness in dialogue systems. In Proceed- ings of the 28th International Conference on Com- putational Linguistics, pages 4403-4416, Barcelona, Spain (Online). International Committee on Compu- tational Linguistics. · DOI
  33. Pengfei Liu, Weizhe Yuan, Jinlan Fu, Zhengbao Jiang, Hiroaki Hayashi, and Graham Neubig. 2023. Pre- train, prompt, and predict: A systematic survey of prompting methods in natural language processing. ACM Computing Surveys, 55(9):1-35.
  34. Kaiji Lu, Piotr Mardziel, Fangjing Wu, Preetam Aman- charla, and Anupam Datta. 2020. Gender bias in neural natural language processing. Logic, Language, and Security: Essays Dedicated to Andre Scedrov on the Occasion of His 65th Birthday, pages 189-202.
  35. Ximing Lu, Sean Welleck, Jack Hessel, Liwei Jiang, Lianhui Qin, Peter West, Prithviraj Ammanabrolu, and Yejin Choi. 2022. Quark: Controllable text generation with reinforced unlearning. Advances in neural information processing systems, 35:27591- 27609.
  36. Anne Maass. 1999. Linguistic intergroup bias: Stereo- type perpetuation through language. In Advances in experimental social psychology, volume 31, pages 79-121. Elsevier.
  37. Bodhisattwa Prasad Majumder, Zexue He, and Julian McAuley. 2022. InterFair: Debiasing with natural language feedback for fair interpretable predictions. arXiv preprint arXiv:2210.07440. · arXiv
  38. Justus Mattern, Zhijing Jin, Mrinmaya Sachan, Rada Mihalcea, and Bernhard Schölkopf. 2022. Under- standing stereotypes in language models: Towards robust measurement and zero-shot debiasing. arXiv preprint arXiv:2212.10678. · arXiv
  39. Nicholas Meade, Spandana Gella, Devamanyu Haz- arika, Prakhar Gupta, Di Jin, Siva Reddy, Yang Liu, and Dilek Hakkani-Tür. 2023. Using in-context learning to improve dialogue safety. arXiv preprint arXiv:2302.00871. · arXiv
  40. Katelyn Mei, Sonia Fereidooni, and Aylin Caliskan. 2023. Bias against 93 stigmatized groups in masked language models and downstream sentiment classifi- cation tasks. In Proceedings of the 2023 ACM Confer- ence on Fairness, Accountability, and Transparency, pages 1699-1710.
  41. Pranav Narayanan Venkit, Sanjana Gautam, Ruchi Pan- chanadikar, Ting-Hao Huang, and Shomir Wilson. 2023. Nationality bias in text generation. In Proceed- ings of the 17th Conference of the European Chap- ter of the Association for Computational Linguistics, pages 116-122, Dubrovnik, Croatia. Association for Computational Linguistics.
  42. Changdae Oh, Heeji Won, Junhyuk So, Taero Kim, Yewon Kim, Hosik Choi, and Kyungwoo Song. 2022. Learning fair representation via distributional con- trastive disentanglement. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 1295-1305.
  43. Ndidi A Okeke, Lionel C Howard, Beth Kurtz-Costes, and Stephanie J Rowley. 2009. Academic race stereo- types, academic self-concept, and racial centrality in african american youth. Journal of Black Psychology, 35(3):366-387.
  44. Swetasudha Panda, Ari Kobren, Michael Wick, and Qinlan Shen. 2022. Don't just clean it, proxy clean it: Mitigating bias by proxy in pre-trained models. In Findings of the Association for Computational Linguistics: EMNLP 2022, pages 5073-5085.
  45. SunYoung Park, Kyuri Choi, Haeun Yu, and Youngjoong Ko. 2023. Never too late to learn: Reg- ularizing gender bias in coreference resolution. In Proceedings of the Sixteenth ACM International Con- ference on Web Search and Data Mining, WSDM '23, page 15-23, New York, NY, USA. Association for Computing Machinery. · DOI
  46. Alicia Parrish, Angelica Chen, Nikita Nangia, Vishakh Padmakumar, Jason Phang, Jana Thompson, Phu Mon Htut, and Samuel Bowman. 2022. BBQ: A hand-built bias benchmark for question answering. In Findings of the Association for Computational Linguistics: ACL 2022, pages 2086-2105, Dublin, Ireland. Association for Computational Linguistics. · DOI · Paper page
  47. Rebecca Qian, Candace Ross, Jude Fernandes, Eric Michael Smith, Douwe Kiela, and Adina Williams. 2022. Perturbation augmentation for fairer NLP. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 9496-9521, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
  48. Yusu Qian, Urwa Muaz, Ben Zhang, and Jae Won Hyun. 2019. Reducing gender bias in word-level language models with a gender-equalizing loss function. In Proceedings of the 57th Annual Meeting of the Asso- ciation for Computational Linguistics: Student Re- search Workshop, pages 223-228, Florence, Italy. Association for Computational Linguistics. · DOI
  49. Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. 2019. Language models are unsupervised multitask learners. OpenAI blog, 1(8):9.
  50. Laria Reynolds and Kyle McDonell. 2021. Prompt pro- gramming for large language models: Beyond the few-shot paradigm. In Extended Abstracts of the 2021 CHI Conference on Human Factors in Com- puting Systems, CHI EA '21, New York, NY, USA. Association for Computing Machinery. · DOI · Paper page
  51. Danielle Saunders, Rosie Sallis, and Bill Byrne. 2022. First the worst: Finding better gender translations dur- ing beam search. In Findings of the Association for Computational Linguistics: ACL 2022, pages 3814- 3823, Dublin, Ireland. Association for Computational Linguistics. · DOI
  52. Timo Schick, Sahana Udupa, and Hinrich Schütze. 2021. Self-diagnosis and self-debiasing: A proposal for re- ducing corpus-based bias in nlp. Transactions of the Association for Computational Linguistics, 9:1408- 1424.
  53. Emily Sheng, Kai-Wei Chang, Prem Natarajan, and Nanyun Peng. 2020. Towards Controllable Biases in Language Generation. In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 3239-3254, Online. Association for Computational Linguistics. · DOI
  54. Emily Sheng, Kai-Wei Chang, Prem Natarajan, and Nanyun Peng. 2021a. "Nice try, kiddo": Investi- gating ad hominems in dialogue responses. In Pro- ceedings of the 2021 Conference of the North Amer- ican Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 750-767, Online. Association for Computational Lin- guistics. · DOI
  55. Emily Sheng, Kai-Wei Chang, Prem Natarajan, and Nanyun Peng. 2021b. Societal biases in language generation: Progress and challenges. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 4275-4293, Online. Association for Computational Linguistics. · DOI · Paper page
  56. Tony Sun, Kellie Webster, Apu Shah, William Yang Wang, and Melvin Johnson. 2021. They, them, theirs: Rewriting with gender-neutral english. arXiv preprint arXiv:2102.06788. · arXiv
  57. Ewoenam Kwaku Tokpo and Toon Calders. 2022. Text style transfer for bias mitigation using masked lan- guage modeling. In Proceedings of the 2022 Con- ference of the North American Chapter of the Asso- ciation for Computational Linguistics: Human Lan- guage Technologies: Student Research Workshop, pages 163-171, Hybrid: Seattle, Washington + On- line. Association for Computational Linguistics. · DOI
  58. Eva Vanmassenhove, Chris Emmery, and Dimitar Shteri- onov. 2021. NeuTral Rewriter: A rule-based and neu- ral approach to automatic rewriting into gender neu- tral alternatives. In Proceedings of the 2021 Confer- ence on Empirical Methods in Natural Language Pro- cessing, pages 8940-8948, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics. · DOI · Paper page
  59. Kellie Webster, Xuezhi Wang, Ian Tenney, Alex Beutel, Emily Pitler, Ellie Pavlick, Jilin Chen, Ed Chi, and Slav Petrov. 2020. Measuring and reducing gendered correlations in pre-trained models. arXiv preprint arXiv:2010.06032. · arXiv
  60. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. 2022. Chain-of-thought prompting elicits rea- soning in large language models. Advances in Neural Information Processing Systems, 35:24824-24837.
  61. Laura Weidinger, Jonathan Uesato, Maribeth Rauh, Conor Griffin, Po-Sen Huang, John Mellor, Amelia Glaese, Myra Cheng, Borja Balle, Atoosa Kasirzadeh, Courtney Biles, Sasha Brown, Zac Kenton, Will Hawkins, Tom Stepleton, Abeba Birhane, Lisa Anne Hendricks, Laura Rimell, William Isaac, Julia Haas, Sean Legassick, Geoffrey Irving, and Iason Gabriel. 2022. Taxonomy of risks posed by language models. In Proceedings of the 2022 ACM Conference on Fair- ness, Accountability, and Transparency, FAccT '22, page 214-229, New York, NY, USA. Association for Computing Machinery. · DOI · Paper page
  62. Tae-Jin Woo, Woo-Jeoung Nam, Yeong-Joon Ju, and Seong-Whan Lee. 2023. Compensatory debiasing for gender imbalances in language models. In ICASSP 2023-2023 IEEE International Conference on Acous- tics, Speech and Signal Processing (ICASSP), pages 1-5. IEEE.
  63. Ke Yang, Charles Yu, Yi R Fung, Manling Li, and Heng Ji. 2023. Adept: A debiasing prompt framework. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pages 10780-10788.
  64. Charles Yu, Sullam Jeoung, Anish Kasi, Pengfei Yu, and Heng Ji. 2023. Unlearning bias in language models by partitioning gradients. In Findings of the Associa- tion for Computational Linguistics: ACL 2023, pages 6032-6048.
  65. Abdelrahman Zayed, Prasanna Parthasarathi, Gonçalo Mordido, Hamid Palangi, Samira Shabanian, and Sarath Chandar. 2023. Deep learning on a healthy data diet: Finding important examples for fairness. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pages 14593-14601.
  66. Zihao Zhao, Eric Wallace, Shi Feng, Dan Klein, and Sameer Singh. 2021. Calibrate before use: Improv- ing few-shot performance of language models. In In- ternational Conference on Machine Learning, pages 12697-12706.
  67. PMLR.
  68. Chujie Zheng, Pei Ke, Zheng Zhang, and Minlie Huang. 2023. Click: Controllable text generation with se- quence likelihood contrastive learning. In Findings of the Association for Computational Linguistics: ACL 2023, pages 1022-1040, Toronto, Canada. Associa- tion for Computational Linguistics. · DOI

Metadata

Authors
Gallegos, I. O., Rossi, R. A., Barrow, J., Tanjim, M. M., Yu, T., Deilamsalehy, H., Zhang, R., Kim, S., & Dernoncourt, F.
Affiliations
  • Stanford University (US)
  • Adobe Systems (United States) (US)
ORCID
Venue
arXiv
Publisher
arXiv
Published
2024
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.2402.01981
arXiv
2402.01981
OpenAlex
W4391591849
Cited by
3
References
0
Citations by year
2026: 1 · 2025: 2
Type
posted content
Language
en
Open access
Licence
Locations
  • arXiv (Cornell University) (submittedVersion, cc-by)
  • arXiv (Cornell University)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
  • Physical Sciences › Computer Science › Artificial Intelligence › Hate Speech and Cyberbullying Detection
Keywords
large language models, social stereotypes, bias mitigation, zero-shot learning

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review