On Second Thought, Let’s Not Think Step by Step! Bias and Toxicity in Zero-Shot Reasoning
Open accessAbstract
Generating a Chain of Thought (CoT) has been shown to consistently improve large language model (LLM) performance on a wide range of NLP tasks. However, prior work has mainly focused on logical reasoning tasks (e.g. arithmetic, commonsense QA); it remains unclear whether improvements hold for more diverse types of reasoning, especially in socially situated contexts. Concretely, we perform a controlled evaluation of zero-shot CoT across two socially sensitive domains: harmful questions and stereotype benchmarks. We find that zero-shot CoT reasoning in sensitive domains significantly increases a model's likelihood to produce harmful or undesirable output, with trends holding across different prompt formats and model variants. Furthermore, we show that harmful CoTs increase with model size, but decrease with improved instruction following. Our work suggests that zero-shot CoT should be used with caution on socially important tasks, especially when marginalized groups or sensitive topics are involved.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Prompting approaches can reduce the sensitivity of CoT explanations to input perturbations and stereotypes, which our findings on prompting for debiasing corroborate.
“Prompting approaches can reduce the sensitivity of CoT explanations to input perturbations and stereotypes (Shaikh et al., 2022;Ganguli et al., 2023;Shi et al., 2023), which our findings on prompting for debiasing corroborate.”
From Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Prompting (2023) · cites this paper as Shaikh et al., 2022
Section 5.4, “Prompting with Instruction Mitigations” (page unavailable), reports that an explicit instruction reduced CoT’s adverse effect on stereotype-benchmark accuracy for TD3. It does not establish that prompting reduces the sensitivity of CoT explanations to input perturbations; it measures benchmark outcomes rather than that property of explanations. (LLM, conf=0.85)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Glove: Global Vectors for Word Representation
Pennington, J., Socher, R., & Manning, C. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1162 · Cited by 34,071 · see citing works
LLaMA: Open and Efficient Foundation Language Models
Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.-A., Lacroix, T., Rozière, B., Goyal, N., Hambro, E., Azhar, F., et al. (2023) · arXiv · doi:10.48550/arxiv.2302.13971 · Cited by 4,602 · see citing works
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
Wei, J., Wang, X., Schuurmans, D., Bosma, M., Ichter, B., Xia, F., Chi, E., Le, Q., & Zhou, D. (2022) · arXiv · doi:10.48550/arxiv.2201.11903 · Cited by 4,263 · see citing works
Training language models to follow instructions with human feedback
Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C. L., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., et al. (2022) · arXiv · doi:10.48550/arxiv.2203.02155 · Cited by 4,208 · see citing works
Semantics derived automatically from language corpora contain human-like biases
Caliskan, A., Bryson, J. J., & Narayanan, A. (2017) · Science · doi:10.1126/science.aal4230 · Cited by 2,887 · see citing works
Large language models are zero-shot reasoners
Kojima, T., Gu, S. (., Reid, M., Matsuo, Y., & Iwasawa, Y. (2022) · Advances in Neural Information Processing Systems 35 · doi:10.52202/068431-1613 · Cited by 2,013 · see citing works
Scaling Instruction-Finetuned Language Models
Chung, H. W., Hou, L., Longpre, S., Zoph, B., Tay, Y., Fedus, W., Li, Y., Wang, X., Dehghani, M., Brahma, S., et al. (2022) · arXiv · doi:10.48550/arxiv.2210.11416 · Cited by 1,441 · see citing works
Making Pre-trained Language Models Better Few-shot Learners
Gao, T., Fisch, A., & Chen, D. (2021) · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.18653/v1/2021.acl-long.295 · Cited by 1,380 · see citing works
Finetuned Language Models Are Zero-Shot Learners
Wei, J., Bosma, M., Zhao, V. Y., Guu, K., Yu, A. W., Lester, B., Du, N., Dai, A. M., & Le, Q. V. (2021) · arXiv · doi:10.48550/arxiv.2109.01652 · Cited by 1,101 · see citing works
TruthfulQA: Measuring How Models Mimic Human Falsehoods
Lin, S., Hilton, J., & Evans, O. (2022) · Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2022.acl-long.229 · Cited by 904 · see citing works
Self-Consistency Improves Chain of Thought Reasoning in Language Models
Wang, X., Wei, J., Schuurmans, D., Le, Q., Chi, E., Narang, S., Chowdhery, A., & Zhou, D. (2022) · arXiv · doi:10.48550/arxiv.2203.11171 · Cited by 829 · see citing works
Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?
Min, S., Lyu, X., Holtzman, A., Artetxe, M., Lewis, M., Hajishirzi, H., & Zettlemoyer, L. (2022) · Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2022.emnlp-main.759 · Cited by 755 · see citing works
RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Models
Gehman, S., Gururangan, S., Sap, M., Choi, Y., & Smith, N. A. (2020) · Findings of the Association for Computational Linguistics: EMNLP 2020 · doi:10.18653/v1/2020.findings-emnlp.301 · Cited by 693 · see citing works
Gender bias in coreference resolution: Evaluation and debiasing methods
Zhao, J., Wang, T., Yatskar, M., Ordonez, V., & Chang, K.-W. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers) · doi:10.18653/v1/n18-2003 · Cited by 684 · see citing works
StereoSet: Measuring stereotypical bias in pretrained language models
Nadeem, M., Bethke, A., & Reddy, S. (2021) · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.18653/v1/2021.acl-long.416 · Cited by 643 · see citing works
Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models
Srivastava, A., Rastogi, A., Rao, A., Shoeb, A. A. M., Abid, A., Fisch, A., Brown, A. R., Santoro, A., Gupta, A., Garriga-Alonso, A., et al. (2022) · arXiv · doi:10.48550/arxiv.2206.04615 · Cited by 569 · see citing works
Calibrate Before Use: Improving Few-Shot Performance of Language Models
Zhao, T. Z., Wallace, E., Feng, S., Klein, D., & Singh, S. (2021) · arXiv · doi:10.48550/arxiv.2102.09690 · Cited by 513 · see citing works
Holistic Evaluation of Language Models
Bommasani, R., Liang, P., & Lee, T. (2023) · Annals of the New York Academy of Sciences · doi:10.1111/nyas.15007 · Cited by 480 · see citing works
CrowS-Pairs: A Challenge Dataset for Measuring Social Biases in Masked Language Models
Nangia, N., Vania, C., Bhalerao, R., & Bowman, S. R. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.154 · Cited by 446 · see citing works
Gender Bias in Coreference Resolution
Rudinger, R., Naradowsky, J., Leonard, B., & Van Durme, B. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers) · doi:10.18653/v1/n18-2002 · Cited by 443 · see citing works
(2021) · International Conference on Learning Representations · Cited by 435 · see citing works
Racial Bias in Hate Speech and Abusive Language Detection Datasets
Davidson, T., Bhattacharya, D., & Weber, I. (2019) · Proceedings of the Third Workshop on Abusive Language Online · doi:10.18653/v1/w19-3504 · Cited by 414 · see citing works
Prompting GPT-3 To Be ReliablePreprint
Si, C., Gan, Z., Yang, Z.-Y., Wang, S.-H., Wang, J., Boyd-Graber, J. L., & Wang, L.-J. (2022) · International Conference on Learning Representations · doi:10.48550/arxiv.2210.09150 · Cited by 395 · see citing works
Least-to-Most Prompting Enables Complex Reasoning in Large Language Models
Zhou, D., Schärli, N., Hou, L., Wei, J., Scales, N., Wang, X., Schuurmans, D., Cui, C., Bousquet, O., Le, Q., et al. (2022) · arXiv · doi:10.48550/arxiv.2205.10625 · Cited by 380 · see citing works
e-SNLI: Natural language inference with natural language explanations
Camburu, O.-M., Rocktäschel, T., Lukasiewicz, T., & Blunsom, P. (2018) · arXiv · doi:10.48550/arxiv.1812.01193 · Cited by 367 · see citing works
Red Teaming Language Models with Language Models
Perez, E., Huang, S., Song, F., Cai, T., Ring, R., Aslanides, J., Glaese, A., McAleese, N., & Irving, G. (2022) · Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2022.emnlp-main.225 · Cited by 363 · see citing works
Turpin, M., Michael, J., Perez, E., & Bowman, S. R. (2023) · arXiv · doi:10.48550/arxiv.2305.04388 · Cited by 262 · see citing works
Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Them
Suzgun, M., Scales, N., Schärli, N., Gehrmann, S., Tay, Y., Chung, H. W., Chowdhery, A., Le, Q., Chi, E., Zhou, D., et al. (2023) · Findings of the Association for Computational Linguistics: ACL 2023 · doi:10.18653/v1/2023.findings-acl.824 · Cited by 253 · see citing works
Gonen, H., & Goldberg, Y. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1903.03862 · Cited by 242 · see citing works
Positive Stereotypes Are Pervasive and Powerful
Czopp, A. M., Kay, A. C., & Cheryan, S. (2015) · Perspectives on Psychological Science · doi:10.1177/1745691615588091 · Cited by 232 · see citing works
True Few-Shot Learning with Language Models
Perez, E., Kiela, D., & Cho, K. (2021) · arXiv · doi:10.48550/arxiv.2105.11447 · Cited by 226 · see citing works
UL2: Unifying Language Learning ParadigmsPreprint
Tay, Y., Dehghani, M., Tran, V. Q., Garcia, X., Wei, J., Wang, X., Chung, H. W., Shakeri, S., Bahri, D., Schuster, T., et al. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2205.05131 · Cited by 194 · see citing works
BBQ: A hand-built bias benchmark for question answering
Parrish, A., Chen, A., Nangia, N., Padmakumar, V., Phang, J., Thompson, J., Htut, P. M., & Bowman, S. (2022) · Findings of the Association for Computational Linguistics: ACL 2022 · doi:10.18653/v1/2022.findings-acl.165 · Cited by 186 · see citing works
Automatically Identifying Words That Can Serve as Labels for Few-Shot Text Classification
Schick, T., Schmid, H., & Schütze, H. (2020) · Proceedings of the 28th International Conference on Computational Linguistics · doi:10.18653/v1/2020.coling-main.488 · Cited by 184 · see citing works
Stereotyping Norwegian Salmon: An Inventory of Pitfalls in Fairness Benchmark Datasets
Blodgett, S. L., Lopez, G., Olteanu, A., Sim, R., & Wallach, H. (2021) · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.18653/v1/2021.acl-long.81 · Cited by 147 · cited 1× · see citing works
“Bench-marks often-time have differing conceptualizations of bias (Blodgett et al., 2021), leading to contra-dictory results (Delobelle et al., 2022; Cao et al., 2022; Goldfarb-Tarrant et al., 2021).”
BackgroundRed Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learned
Ganguli, D., Lovitt, L., Kernion, J., Askell, A., Bai, Y., Kadavath, S., Mann, B., Perez, E., Schiefer, N., Ndousse, K., et al. (2022) · arXiv · doi:10.48550/arxiv.2209.07858 · Cited by 130 · see citing works
An Empirical Survey of the Effectiveness of Debiasing Techniques for Pre-trained Language Models
Meade, N., Poole-Dayan, E., & Reddy, S. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.132 · Cited by 116 · see citing works
Exploiting Programmatic Behavior of LLMs: Dual-Use Through Standard Security Attacks
Kang, D., Li, X., Stoica, I., Guestrin, C., Zaharia, M., & Hashimoto, T. (2024) · 2024 IEEE Security and Privacy Workshops (SPW) · doi:10.1109/spw63631.2024.00018 · Cited by 114 · see citing works
Process for Adapting Language Models to Society (PALMS) with Values-Targeted Datasets
Solaiman, I., & Dennison, C. (2021) · arXiv · doi:10.48550/arxiv.2106.10328 · Cited by 90 · see citing works
What learning algorithm is in-context learning? Investigations with linear models
Akyürek, E., Schuurmans, D., Andreas, J., Ma, T., & Zhou, D. (2022) · arXiv · doi:10.48550/arxiv.2211.15661 · Cited by 86 · see citing works
Ask Me Anything: A simple strategy for prompting language modelsPreprint
Arora, S., Narayan, A., Chen, M. F., Orr, L., Guha, N., Bhatia, K., Chami, I., Sala, F., & Ré, C. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2210.02441 · Cited by 84 · see citing works
Language Models are Multilingual Chain-of-Thought Reasoners
Shi, F., Suzgun, M., Freitag, M., Wang, X., Srivats, S., Vosoughi, S., Chung, H. W., Tay, Y., Ruder, S., Zhou, D., et al. (2022) · arXiv · doi:10.48550/arxiv.2210.03057 · Cited by 60 · see citing works
Delobelle, P., Tokpo, E., Calders, T., & Berendt, B. (2022) · Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/2022.naacl-main.122 · Cited by 57 · see citing works
Ramamurthy, R., Ammanabrolu, P., Brantley, K., Hessel, J., Sifa, R., Bauckhage, C., Hajishirzi, H., & Choi, Y. (2022) · arXiv · doi:10.48550/arxiv.2210.01241 · Cited by 56 · see citing works
The Capacity for Moral Self-Correction in Large Language Models
Ganguli, D., Askell, A., Schiefer, N., Liao, T. I., Lukošiūtė, K., Chen, A., Goldie, A., Mirhoseini, A., Olsson, C., Hernandez, D., et al. (2023) · arXiv · doi:10.48550/arxiv.2302.07459 · Cited by 53 · see citing works
Cao, Y., Pruksachatkun, Y., Chang, K.-W., Gupta, R., Kumar, V., Dhamala, J., & Galstyan, A. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-short.62 · Cited by 44 · see citing works
Intrinsic bias metrics do not correlate with application bias
Goldfarb-Tarrant, S., Marchant, R., Muñoz Sánchez, R., Pandya, M., & Lopez, A. (2021) · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.18653/v1/2021.acl-long.150 · Cited by 33 · see citing works
Inverse Scaling Can Become U-Shaped
Wei, J., Kim, N., Tay, Y., & Le, Q. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2023.emnlp-main.963 · Cited by 25 · see citing works
Prystawski, B., Thibodeau, P. H., Potts, C., & Goodman, N. D. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2209.08141 · Cited by 24 · see citing works
Toward Gender-Inclusive Coreference Resolution
Cao, Y. T., & Daumé, H., III (2020) · Annual Meeting of the Association for Computational Linguistics (ACL) · doi:10.18653/v1/2020.acl-main.418 · Cited by 22 · see citing works
Robustness of Demonstration-based Learning Under Limited Data Scenario
Zhang, H., Zhang, Y., Zhang, R., & Yang, D. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.116 · Cited by 10 · see citing works
Self-citationSrivastava, H. (2022) · Workshop on Cognitive Modeling and Computational Linguistics (CMCL) · doi:10.18653/v1/2022.cmcl-1.11 · Cited by 6 · see citing works
OPT: Open Pre-trained Transformer Language Models
Zhang, S., Roller, S., Goyal, N., Artetxe, M., Chen, M., Chen, S., Dewan, C., Diab, M., Li, X., Lin, X. V., et al. (2022) · arXiv · doi:10.48550/arxiv.2205.01068
BLOOM: A 176B-Parameter Open-Access Multilingual Language Model
Workshop, B., :, Scao, T. L., Fan, A., Akiki, C., Pavlick, E., Ilić, S., Hesslow, D., Castagné, R., Luccioni, A. S., et al. (2022) · arXiv · doi:10.48550/arxiv.2211.05100
Flagged first, then by citations
2 of the 56 references couldn’t be matched to a known paper.
Research works printed but not linked: 3
- Tolga Bolukbasi, Kai-Wei Chang, James Y Zou, Venkatesh Saligrama, and Adam T Kalai. 2016. Man is to computer programmer as woman is to home- maker? debiasing word embeddings. Advances in neural information processing systems, 29.
- Xi Ye and Greg Durrett. 2022. The unreliability of explanations in few-shot in-context learning. arXiv preprint arXiv:2205.03401. · arXiv
- PMLR.
Cited by
Sorted by citations
A Comprehensive Overview of Large Language Models
Naveed, H., Khan, A. U., Qiu, S., Saqib, M., Anwar, S., Usman, M., Akhtar, N., Barnes, N., & Mian, A. (2025) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3744746 · Cited by 1,257 · see citing works
A survey on large language model (LLM) security and privacy: The Good, The Bad, and The Ugly
Yao, Y., Duan, J., Xu, K., Cai, Y., Sun, Z., & Zhang, Y. (2024) · High-Confidence Computing · doi:10.1016/j.hcc.2024.100211 · Cited by 1,083 · see citing works
The future landscape of large language models in medicine
Clusmann, J., Kolbinger, F. R., Muti, H. S., Carrero, Z. I., Eckardt, J.-N., Laleh, N. G., Löffler, C. M. L., Schwarzkopf, S.-C., Unger, M., Veldhuizen, G. P., et al. (2023) · Communications Medicine · doi:10.1038/s43856-023-00370-1 · Cited by 1,045 · see citing works
Can Large Language Models Transform Computational Social Science?
Ziems, C., Held, W., Shaikh, O., Chen, J., Zhang, Z., & Yang, D. (2024) · Computational Linguistics · doi:10.1162/coli_a_00502 · Cited by 585 · see citing works
Self-citationZack, T., Lehman, E., Suzgun, M., Rodriguez, J. A., Celi, L. A., Gichoya, J., Jurafsky, D., Szolovits, P., Bates, D. W., Abdulnour, R.-E. E., et al. (2024) · The Lancet Digital Health · doi:10.1016/s2589-7500(23)00225-x · Cited by 510 · see citing works
Plan-and-Solve Prompting: Improving Zero-Shot Chain-of-Thought Reasoning by Large Language Models
Wang, L., Xu, W., Lan, Y., Hu, Z., Lan, Y., Lee, R. K.-W., & Lim, E.-P. (2023) · Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2023.acl-long.147 · Cited by 344 · see citing works
Shen, X., Chen, Z., Backes, M., Shen, Y., & Zhang, Y. (2024) · Proceedings of the 2024 on ACM SIGSAC Conference on Computer and Communications Security · doi:10.1145/3658644.3670388 · Cited by 441 · see citing works
AI agents under threat: A survey of key security challenges and future pathways
Deng, Z., Guo, Y., Han, C., Ma, W., Xiong, J., Wen, S., & Xiang, Y. (2025) · ACM Computing Surveys · doi:10.1145/3716628 · Cited by 210 · see citing works
Feng, S., Park, C. Y., Liu, Y., & Tsvetkov, Y. (2023) · Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2023.acl-long.656 · Cited by 149 · see citing works
Reasoning with Language Model Prompting: A Survey
Qiao, S., Ou, Y., Zhang, N., Chen, X., Yao, Y., Deng, S., Tan, C., Huang, F., & Chen, H. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.294 · Cited by 218 · see citing works
Safeguarding large language models: a survey
Dong, Y., Mu, R., Zhang, Y., Sun, S., Zhang, T., Wu, C., Jin, G., Qi, Y., Hu, J., Meng, J., et al. (2025) · Artificial Intelligence Review · doi:10.1007/s10462-025-11389-2 · Cited by 161 · see citing works
Perils and opportunities in using large language models in psychological research
Abdurahman, S., Atari, M., Karimi-Malekabadi, F., Xue, M. J., Trager, J., Park, P. S., Golazizian, P., Omrani, A., & Dehghani, M. (2024) · PNAS Nexus · doi:10.1093/pnasnexus/pgae245 · Cited by 111 · see citing works
Language Model Behavior: A Comprehensive Survey
Chang, T. A., & Bergen, B. K. (2024) · Computational Linguistics · doi:10.1162/coli_a_00492 · Cited by 105 · see citing works
Wan, P., Huang, Z., Tang, W., Nie, Y., Pei, D., Deng, S., Chen, J., Zhou, Y., Duan, H., Chen, Q., et al. (2024) · Nature Medicine · doi:10.1038/s41591-024-03148-7 · Cited by 97 · see citing works
Turpin, M., Michael, J., Perez, E., & Bowman, S. R. (2023) · arXiv · doi:10.48550/arxiv.2305.04388 · Cited by 262 · see citing works
Pan, L., Saxon, M., Xu, W., Nathani, D., Wang, X., & Wang, W. Y. (2024) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00660 · Cited by 86 · see citing works
Schulhoff, S., Pinto, J., Khan, A., Bouchard, L.-F., Si, C., Anati, S., Tagliabue, V., Kost, A., Carnahan, C., & Boyd-Graber, J. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2023.emnlp-main.302 · Cited by 54 · see citing works
From Classification to Clinical Insights
Englhardt, Z., Ma, C., Morris, M. E., Chang, C.-C., Xu, X., Qin, L., McDuff, D., Liu, X., Patel, S., & Iyer, V. (2024) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3659604 · Cited by 66 · see citing works
Zhang, Z., Yao, Y., Zhang, A., Tang, X., Ma, X., He, Z., Wang, Y., Gerstein, M., Wang, R., Liu, G., et al. (2025) · ACM Computing Surveys · doi:10.1145/3719341 · Cited by 51 · see citing works
Unique Security and Privacy Threats of Large Language Models: A Comprehensive Survey
Wang, S., Zhu, T., Liu, B., Ding, M., Ye, D., Zhou, W., & Yu, P. S. (2025) · ACM Computing Surveys · doi:10.1145/3764113 · Cited by 64 · see citing works
Bhattacharya, M., Pal, S., Chatterjee, S., Lee, S., & Chakraborty, C. (2024) · Molecular Therapy — Nucleic Acids · doi:10.1016/j.omtn.2024.102255 · Cited by 38 · see citing works
On large language models safety, security, and privacy: A survey
Zhang, R., Li, H., Qian, X., Jiang, W., & Chen, H.-X. (2025) · Journal of Electronic Science and Technology · doi:10.1016/j.jnlest.2025.100301 · Cited by 45 · see citing works
Assessing and alleviating state anxiety in large language models
Ben‐Zion, Z., Witte, K., Jagadish, A. K., Duek, O. A., Harpaz‐Rotem, I., Khorsandian, M.-C., Burrer, A., Seifritz, E., Homan, P., Schulz, E., et al. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-01512-6 · Cited by 46 · see citing works
He, J., Wang, L., Hu, Y., Liu, N., Liu, H., Xu, X., & Shen, H. T. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01785 · Cited by 40 · see citing works
Enhancing scientific table understanding with type-guided chain-of-thought
Yin, Z., & Wang, S. (2025) · Information Processing & Management · doi:10.1016/j.ipm.2025.104159 · Cited by 28 · see citing works
Fagbohun, O., Harrison, R. M., & Dereventsov, A. (2023) · Journal of Artificial Intelligence Machine Learning and Data Science · doi:10.51219/jaimld/oluwole-fagbohun/15 · Cited by 43 · see citing works
Esmradi, A., Yip, D. W., & Chan, C. F. (2024) · Communications in Computer and Information Science · doi:10.1007/978-981-97-1274-8_6 · Cited by 24 · see citing works
LangTest: A comprehensive evaluation library for custom LLM and NLP models
Nazir, A., Chakravarthy, T. K., Cecchini, D., Khajuria, R., Sharma, P., Mirik, A. T., Kocaman, V., & Talby, D. (2024) · Software Impacts · doi:10.1016/j.simpa.2024.100619 · Cited by 22 · see citing works
Fan, X., Xiao, Q., Zhou, X., Pei, J., Sap, M., Lu, Z., & Shen, H. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713477 · Cited by 23 · see citing works
Lahoti, P., Blumm, N., Ma, X., Kotikalapudi, R., Potluri, S., Tan, Q., Srinivasan, H., Packer, B., Beirami, A., Beutel, A., et al. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.643 · Cited by 24 · see citing works
Mapping the individual, social and biospheric impacts of Foundation Models
Hernández, A. D., Krishna, S., Perini, A., Katell, M. A., Bennett, S., Borda, A., Hashem, Y., Hadjiloizou, S., Mahomed, S., Jayadeva, S., et al. (2024) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3630106.3658939 · Cited by 20 · see citing works
What Is Waiting for Us at the End? Inherent Biases of Game Story Endings in Large Language Models
Taveekitworachai, P., Abdullah, F., Gursesli, M. C., Dewantoro, M. F., Chen, S., Lanatà, A., Guazzini, A., & Thawonmas, R. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-47658-7_26 · Cited by 16 · see citing works
Yip, D. W., Esmradi, A., & Chan, C. F. (2023) · IEEE Asia-Pacific Conference on Computer Science and Data Engineering (CSDE) · doi:10.1109/csde59766.2023.10487667 · Cited by 19 · see citing works
Arabic Fake News Dataset Development: Humans and AI-Generated Contributions
Himdi, H., Zamzami, N., Najar, F., Alrehaili, M., & Bouguila, N. (2025) · IEEE Access · doi:10.1109/access.2025.3556376 · Cited by 17 · see citing works
FactKB: Generalizable Factuality Evaluation using Language Models Enhanced with Factual Knowledge
Feng, S., Balachandran, V., Bai, Y., & Tsvetkov, Y. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.59 · Cited by 21 · see citing works
Han, H. (2024) · Ethics & Behavior · doi:10.1080/10508422.2024.2347661 · Cited by 14 · see citing works
Guo, Q., Guo, Y., & Zhao, J. (2024) · Neural Computing and Applications · doi:10.1007/s00521-024-09728-5 · Cited by 9 · see citing works
Perils and Opportunities in Using Large Language Models in Psychological ResearchPreprint
Abdurahman, S., Atari, M., Karimi-Malekabadi, F., Xue, M. J., Trager, J. P., Park, P. S., Golazizian, P., Omrani, A., & Dehghani, M. (2023) · doi:10.31219/osf.io/tg79n · Cited by 8 · see citing works
A Brief Survey on Safety of Large Language Models
Gao, Z., Liu, X., Lan, Y.-S., & Yang, Z. (2024) · Journal of Computing and Information Technology · doi:10.20532/cit.2024.1005778 · Cited by 10 · see citing works
Explainable and Interactive LLMs-Augmented Depression Detection in Social Media
Qin, W., Chen, Z., Yang, X., Wang, L., Lan, Y., Ren, W., & Hong, R. (2025) · IEEE Transactions on Computational Social Systems · doi:10.1109/tcss.2025.3591755 · Cited by 9 · see citing works
SocialStigmaQA: A Benchmark to Uncover Stigma Amplification in Generative Language Models
Nagireddy, M., Chiazor, L., Singh, M., & Baldini, I. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i19.30142 · Cited by 8 · see citing works
Perils and Opportunities in Using Large Language Models in Psychological ResearchPreprint
Abdurahman, S., Atari, M., Karimi-Malekabadi, F., Xue, M. J., Trager, J. P., Park, P. S., Golazizian, P., Omrani, A., & Dehghani, M. (2023) · doi:10.31234/osf.io/d695y · Cited by 8 · see citing works
Kikuchi, T., Nakao, T., Nakamura, Y., Hanaoka, S., Mori, H., & Yoshikawa, T. (2024) · American Journal of Neuroradiology · doi:10.3174/ajnr.a8332 · Cited by 7 · see citing works
CoSafe: Evaluating Large Language Model Safety in Multi-Turn Dialogue Coreference
Yu, E., Li, J., Liao, M., Wang, S., Gao, Z., Mi, F., & Hong, L. (2024) · arXiv · doi:10.48550/arxiv.2406.17626 · Cited by 7 · see citing works
“HH-RLHF (Ganguli et al., 2022) 38 , 961 Dialogue Analyse how people red-team LLMs. HarmfulQ (Shaikh et al., 2023) 200 Prompt Machine-written prompts to evaluate LLM safety.”
BackgroundLiu, R., Geng, J., Wu, A. J., Sucholutsky, I., Lombrozo, T., & Griffiths, T. L. (2024) · arXiv · doi:10.48550/arxiv.2410.21333 · Cited by 6 · see citing works
“Furthermore, CoT can also increase the frequency of harmful outputs (Shaikh et al., 2023).”
A safety realignment framework via subspace-oriented model fusion for large language models
Yi, X., Zheng, S., Wang, L., Wang, X., & He, L. (2024) · Knowledge-Based Systems · doi:10.1016/j.knosys.2024.112701 · Cited by 10 · see citing works
Yi, X., Li, Y., Zheng, S., Wang, L., Wang, X., & He, L. (2025) · Knowledge-Based Systems · doi:10.1016/j.knosys.2025.114295 · Cited by 5 · see citing works
A Survey on Stereotype Detection in Natural Language Processing
Cignarella, A. T., Giachanou, A., & Lefever, E. (2025) · ACM Computing Surveys · doi:10.1145/3770754 · Cited by 7 · see citing works
The state as a model for AI control and alignment
Elsner, M. (2024) · AI & Society · doi:10.1007/s00146-024-02063-2 · Cited by 4 · see citing works
Large Language Models (LLMs): A systematic study in Administration and Business
Pessanha, G. R. G., Vieira, A. G., & Brandão, W. C. (2024) · RAM. Revista de Administração Mackenzie · doi:10.1590/1678-6971/eramd240059 · Cited by 5 · see citing works
Yang, Z., Tullo, D., & Rabbany, R. (2025) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3711896.3737271 · Cited by 3 · see citing works
Political Bias of Large Language Models in Few-Shot News Summarization
Onishi, T., & Caverlee, J. (2024) · Communications in computer and information science · doi:10.1007/978-3-031-71975-2_3 · Cited by 3 · see citing works
Gender Trouble in Language Models: An Empirical Audit Guided by Gender Performativity Theory
Hafner, F. S., Valdivia, A., & Rocher, L. (2025) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3715275.3732112 · Cited by 3 · see citing works
Improving hate speech detection with large language models
Umansky, N., Kubli, M., Kotarcic, A., Bronner, L., Kurer, S., Grech, P., Hangartner, D., Gilardi, F., & Donnay, K. (2026) · European Journal of Political Research · doi:10.1017/s1475676525100546 · Cited by 2 · see citing works
GenieWizard: Multimodal App Feature Discovery with Large Language Models
Yang, J., Shi, Y., Gu, C., Zheng, Z., Jain, A., Li, T., Lam, M. S., & Landay, J. A. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3714327 · Cited by 4 · see citing works
Enhancing Hate Speech Detection with Fine-Tuned Large Language Models Requires High-Quality DataPreprint
Umansky, N., Kubli, M., Donnay, K., Gilardi, F., Hangartner, D., Kotarcic, A., Bronner, L., Kurer, S., & Grech, P. (2024) · doi:10.31219/osf.io/7kbqt · Cited by 2 · see citing works
SafeText: A Unified Approach for Detecting and Mitigating Toxicity and Bias in Textual Data
Agarwal, A., Bera, A., & De, T. (2025) · Lecture notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering · doi:10.1007/978-3-032-05832-4_5 · Cited by 1 · see citing works
Small, W. R., Crowley, R. J., Pariente, C. S., Zhang, J., Eaton, K. P., Jiang, L. Y., Oermann, E. K., & Aphinyanaphongs, Y. (2026) · npj Health Systems · doi:10.1038/s44401-025-00059-8 · Cited by 1 · see citing works
Pantelidou, N., Leivada, E., Montero, R., & Morosi, P. (2026) · PLoS ONE · doi:10.1371/journal.pone.0343164 · Cited by 4 · see citing works
Wang, R., He, H., & Sun, Y. (2024) · Lecture notes in computer science · doi:10.1007/978-981-97-5669-8_23 · Cited by 1 · see citing works
MPHDetect: Multi-View Prompting and Hypergraph Fusion for Malevolence Detection in Dialogues
Xu, B., Qiao, X., Lin, H., & Zong, L. (2024) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3627673.3679966 · Cited by 1 · see citing works
Han, H. (2025) · International Journal of Ethics Education · doi:10.1007/s40889-025-00208-5 · Cited by 1 · see citing works
Advances in reasoning by prompting large language models: A survey
Hong, R., Pang, X., & Zhang, C. (2026) · Cybernetics and Intelligence · doi:10.26599/cai.2024.9390004 · Cited by 1 · see citing works
Xu, B., Qiao, X., Zhang, X., Wan, J., Liu, X., Zong, L., & Lin, H. (2024) · IEEE International Conference on Bioinformatics and Biomedicine (BIBM) · doi:10.1109/bibm62325.2024.10822192
Debiasing Large Language Models with Structured Knowledge
Ma, C., Tianyu, Z. H. A. O., & Okumura, M. (2025) · IEICE Transactions on Information and Systems · doi:10.1587/transinf.2024edp7326
AI and Challenges to Personal Data and Data Protection in Research
Lindblom, L. (2025) · CTE · doi:10.3384/9789181181814
Efficient Safety Retrofitting Against Jailbreaking for LLMs
García-Gasulla, D., Tormos, A., Arias-Duart, A., Hinjos, D., Molina-Sedano, O., Gururajan, A. K., & Cardello, M. E. (2025) · Lecture notes in computer science · doi:10.1007/978-3-032-02018-5_39 · Cited by 1 · see citing works
Fang, S., Cheng, K., Zhang, J., Qin, Z., & Zhang, M. (2025) · Lecture notes in computer science · doi:10.1007/978-981-95-3537-8_22
Thoughts Behind Attack: Enhancing Security Against Jailbreak Attacks Using Chain-of-Thought
Tao, Z., Xu, B., Yang, M., Guan, H., Lü, W., Cao, H., Zhu, C., & Zhao, T. (2025) · Lecture notes in computer science · doi:10.1007/978-981-95-3352-7_16
Predicting Player Churn with LLMs: A Comprehensive Evaluation of World Knowledge and Reasoning
Schneider, T., Sparrenberg, L., & Sifa, R. (2025) · IEEE International Conference on Data Science and Advanced Analytics (DSAA) · doi:10.1109/dsaa65442.2025.11248010
Cha, K., & Joung, J. (2026) · Journal of Korean Institute of Industrial Engineers · doi:10.7232/jkiie.2026.52.1.070
Li, M., & Krishnamachari, B. (2026) · IEEE SoutheastCon · doi:10.1109/southeastcon63549.2026.11476431
Debiasing LLMs in Knowledge-Intensive Tasks via Information-Gain Guided Front-Door Adjustment
Miao, X., Li, Y., Kang, H., Xu, M., Wen, J., Zhu, Y. Z., Zhong, M., Jiang, J., & Qian, T. (2026) · Lecture notes in computer science · doi:10.1007/978-981-92-0369-7_24
The More You Say, the More You Risk: Ethical Concerns in Large Language Model Reasoning Frameworks
Zhao, J., Cao, L., Zhang, X., Shi, K., Wang, Y., & Penn, G. (2026) · International World Wide Web Conference (WWW) · doi:10.1145/3774905.3795475
Ju, J., Seo, J., Jang, J., Lee, D., & Park, H. (2026) · IEEE Pacific Visualization Conference (PacificVis) · doi:10.1109/pacificvis68791.2026.00019
Automated bias blind spots: Examining stereotype associations in VLM-as-a-judge paradigms
Hong, R. S., Ovalle, A., Ung, M., Spiliopoulou, E., Sagun, L., Williams, A., & Ross, C. (2026) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3805689.3806459
Mitigating Bias in Large Language Model Based Question Answering through Causal Front Door Prompting
Yang, Y., Xu, Z., Li, J., Ma, C., Chan, J., Sanderson, M., Zheng, X., & Ren, Y. (2026) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3805712.3809679 · Cited by 1 · see citing works
LLM-Based Detection of Tangled Code Changes for Higher-Quality Method-Level Bug Datasets
Opu, M. N. I., Wang, S., & Chowdhury, S. (2026) · International Conference on Mining Software Repositories (MSR) · doi:10.1145/3793302.3793351 · Cited by 3 · see citing works
Jia, L., Zhou, J., He, Y., & Sun, G. (2026) · Communications in computer and information science · doi:10.1007/978-981-92-3994-8_27
Generative AI for Healthcare and Education
Sartori, L., & Piva, R. (2026) · Springer eBooks · doi:10.1007/978-3-031-61050-9_41-1
- Authors
- Shaikh, O., Zhang, H., Held, W., Bernstein, M., & Yang, D.
- Affiliations
- Georgia Institute of Technology (US)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2022
- Pages
- 4454-4470
- ISSN
- 0736-587X (ISSN-L)
- arXiv
- 2212.08061
- OpenAlex
- W4385571830
- Cited by
- 76
- References
- 56
- Citations by year
- 2026: 14 · 2025: 24 · 2024: 22 · 2023: 16
- FWCI
- 19.02
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Proceedings of the conference - Association for Computational Linguistics. Meeting (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- National Science Foundation (2247357)
- Topics
- Social Sciences › Safety Research › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Hate Speech and Cyberbullying Detection
- Keywords
- chain-of-thought prompting, zero-shot reasoning, large language models, bias, toxicity, instruction following, model size effects, marginalized communities
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
