The DOI matches its DataCite record.
Your Large Language Model is Secretly a Fairness Proponent and You Should Prompt it Like One
Open accessAbstract
The widespread adoption of large language models (LLMs) underscores the urgent need to ensure their fairness. However, LLMs frequently present dominant viewpoints while ignoring alternative perspectives from minority parties, resulting in potential biases. We hypothesize that these fairness-violating behaviors occur because LLMs express their viewpoints using a human personality that represents the majority of training data. In response to this, we validate that prompting LLMs with specific roles can allow LLMs to express diverse viewpoints. Building on this insight and observation, we develop FairThinking, a pipeline designed to automatically generate roles that enable LLMs to articulate diverse perspectives for fair expressions. To evaluate FairThinking, we create a dataset with a thousand items covering three fairness-related topics and conduct experiments on GPT-3.5, GPT-4, Llama2, and Mistral to demonstrate its superior performance.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
LLMs are highly sensitive to AI personas or alternative prompt framings that explicitly ask for disagreement, uncertainty and pluralism.
“The encouraging side of this picture is that LLMs are also highly sensitive to AI personas or alternative prompt framings that explicitly ask for disagreement, uncertainty and pluralism.”
From Complacent, Not Sycophantic: Reframing Large Language Models and Designing AI Literacy for Complacent Machines (2026) · cites this paper as [24]
Section 3, “A Closer Look at LLMs’ Biases” (page not specified), reports that role prompts can elicit different viewpoints, and Appendix E reports that human raters found the answers generally matched the assigned roles. The paper also uses multi-role debate to bring in alternative perspectives, but it does not show that prompts explicitly asking for uncertainty produce the claimed sensitivity. (LLM, conf=0.85)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
No references could be resolved yet.
Research works printed but not linked: 24
- Bai, Yuntao, Jones, Andy, Ndousse, Kamal, Askell, Amanda, Chen, Anna, DasSarma, Nova, Drain, Dawn, Fort, Stanislav, Ganguli, Deep, Henighan, Tom, et al. Training a helpful and harmless assistant with reinforce- ment learning from human feedback. arXiv preprint arXiv:2204.05862, 2022. · arXiv
- Chan, Chi-Min, Chen, Weize, Su, Yusheng, Yu, Jianxuan, Xue, Wei, Zhang, Shanghang, Fu, Jie, and Liu, Zhiyuan. Chateval: Towards better llm-based evaluators through multi-agent debate. arXiv preprint arXiv:2308.07201, 2023. · arXiv
- Chowdhery, Aakanksha, Narang, Sharan, Devlin, Jacob, Bosma, Maarten, Mishra, Gaurav, Roberts, Adam, Barham, Paul, Chung, Hyung Won, Sutton, Charles, Gehrmann, Sebastian, et al. Palm: Scaling language modeling with pathways. Journal of Machine Learning Research, 24(240):1-113, 2023.
- Christiano, Paul F, Leike, Jan, Brown, Tom, Martic, Miljan, Legg, Shane, and Amodei, Dario. Deep reinforcement learning from human preferences. Advances in neural information processing systems, 30, 2017.
- Cui, Shiyao, Zhang, Zhenyu, Chen, Yilong, Zhang, Wenyuan, Liu, Tianyun, Wang, Siqi, and Liu, Tingwen. Fft: Towards harmlessness evaluation and analysis for llms with factuality, fairness, toxicity. arXiv preprint arXiv:2311.18580, 2023. · arXiv
- Du, Yilun, Li, Shuang, Torralba, Antonio, Tenenbaum, Joshua B, and Mordatch, Igor. Improving factuality and reasoning in language models through multiagent debate. arXiv preprint arXiv:2305.14325, 2023. · arXiv
- Gallegos, Isabel O, Rossi, Ryan A, Barrow, Joe, Tanjim, Md Mehrab, Kim, Sungchul, Dernoncourt, Franck, Yu, Tong, Zhang, Ruiyi, and Ahmed, Nesreen K. Bias and fairness in large language models: A survey. arXiv preprint arXiv:2309.00770, 2023. · arXiv
- Huang, Yue, Zhang, Qihui, Sun, Lichao, et al. Trustgpt: A benchmark for trustworthy and responsible large lan- guage models. arXiv preprint arXiv:2306.11507, 2023. · arXiv
- Levy, Shahar, Lazar, Koren, and Stanovsky, Gabriel. Col- lecting a large-scale gender bias dataset for corefer- ence resolution and machine translation. arXiv preprint arXiv:2109.03858, 2021. · arXiv
- Li, Tao, Khot, Tushar, Khashabi, Daniel, Sabharwal, Ashish, and Srikumar, Vivek. Unqovering stereotyp- ing biases via underspecified questions. arXiv preprint arXiv:2010.02428, 2020. · arXiv
- Li, Yingji, Du, Mengnan, Song, Rui, Wang, Xin, and Wang, Ying. A survey on fairness in large language models. arXiv preprint arXiv:2308.10149, 2023b. · arXiv
- Li, Yunqi and Zhang, Yongfeng. Fairness of chatgpt. arXiv preprint arXiv:2305.18569, 2023. · arXiv
- Nozza, Debora, Bianchi, Federico, Lauscher, Anne, Hovy, Dirk, et al. Measuring harmful sentence completion in language models for lgbtqia+ individuals. In Proceed- ings of the Second Workshop on Language Technology for Equality, Diversity and Inclusion. Association for Computational Linguistics, 2022.
- Ouyang, Long, Wu, Jeffrey, Jiang, Xu, Almeida, Diogo, Wainwright, Carroll, Mishkin, Pamela, Zhang, Chong, Agarwal, Sandhini, Slama, Katarina, Ray, Alex, et al. Training language models to follow instructions with hu- man feedback. Advances in Neural Information Process- ing Systems, 35:27730-27744, 2022.
- Parrish, Alicia, Chen, Angelica, Nangia, Nikita, Padmaku- mar, Vishakh, Phang, Jason, Thompson, Jana, Htut, Phu Mon, and Bowman, Samuel R. Bbq: A hand-built bias benchmark for question answering. arXiv preprint arXiv:2110.08193, 2021. · arXiv
- Qian, Rebecca, Ross, Candace, Fernandes, Jude, Smith, Eric, Kiela, Douwe, and Williams, Adina. Pertur- bation augmentation for fairer nlp. arXiv preprint arXiv:2205.12586, 2022. · arXiv
- Rafailov, Rafael, Sharma, Archit, Mitchell, Eric, Ermon, Stefano, Manning, Christopher D, and Finn, Chelsea. Di- rect preference optimization: Your language model is se- cretly a reward model. arXiv preprint arXiv:2305.18290, 2023. · arXiv
- Salewski, Leonard, Alaniz, Stephan, Rio-Torto, Isabel, Schulz, Eric, and Akata, Zeynep. In-context imperson- ation reveals large language models' strengths and biases. arXiv preprint arXiv:2305.14930, 2023. · arXiv
- Scao, Teven Le, Fan, Angela, Akiki, Christopher, Pavlick, Ellie, Ilić, Suzana, Hesslow, Daniel, Castagné, Roman, Luccioni, Alexandra Sasha, Yvon, Franc ¸ois, et al. Bloom: A 176b-parameter open-access multilingual language model. arXiv preprint arXiv:2211.05100, 2022. · arXiv
- Touvron, Hugo, Martin, Louis, Stone, Kevin, Albert, Peter, Almahairi, Amjad, Babaei, Yasmine, Bashlykov, Nikolay, Batra, Soumya, Bhargava, Prajjwal, Bhosale, Shruti, et al. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288, 2023. · arXiv
- Wan, Yuxuan, Wang, Wenxuan, He, Pinjia, Gu, Jiazhen, Bai, Haonan, and Lyu, Michael R. Biasasker: Measuring the bias in conversational ai system. In Proceedings of the 31st ACM Joint European Software Engineering Con- ference and Symposium on the Foundations of Software Engineering, pp. 515-527, 2023.
- Wei, Jason, Wang, Xuezhi, Schuurmans, Dale, Bosma, Maarten, Xia, Fei, Chi, Ed, Le, Quoc V, Zhou, Denny, et al. Chain-of-thought prompting elicits reasoning in large language models. Advances in Neural Information Processing Systems, 35:24824-24837, 2022.
- Zhuo, Terry Yue, Huang, Yujin, Chen, Chunyang, and Xing, Zhenchang. Red teaming chatgpt via jailbreaking: Bias, robustness, reliability and toxicity. arXiv preprint arXiv:2301.12867, 2023. · arXiv
- Zou, Andy, Wang, Zifan, Kolter, J Zico, and Fredrik- son, Matt. Universal and transferable adversarial at- tacks on aligned language models. arXiv preprint arXiv:2307.15043, 2023. GPT-3.5-Turbo Men are not necessarily less suited to be a CEO than women. The reasons are shown as follows: 1. Gender does not determine suitability for being a CEO. · arXiv
Cited by
Sorted by citations
Germani, F., & Spitale, G. (2026) · arXiv · doi:10.48550/arxiv.2605.14544 · Cited by 1
Metadata
- Authors
- Li, T., Zhang, X., Du, C., Pang, T., Liu, Q., Guo, Q., Shen, C., & Liu, Y.
- Affiliations
- Nanyang Technological University (SG)
- Xi'an Jiaotong University (CN)
- Agency for Science, Technology and Research (SG)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2024
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2402.12150
- OpenAlex
- W4391988106
- Cited by
- 3
- References
- 0
- Citations by year
- 2025: 3
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- arXiv (Cornell University) (submittedVersion, other-oa)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Hate Speech and Cyberbullying Detection
- Keywords
- large language models, algorithmic fairness, viewpoint diversity, prompting, minority perspectives, bias in language models
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
