Safetywashing: Do AI Safety Benchmarks Actually Measure Safety Progress?
Open accessAbstract
As artificial intelligence systems grow more powerful, there has been increasing interest in "AI safety" research to address emerging and future risks. However, the field of AI safety remains poorly defined and inconsistently measured, leading to confusion about how researchers can contribute. This lack of clarity is compounded by the unclear relationship between AI safety benchmarks and upstream general capabilities (e.g., general knowledge and reasoning). To address these issues, we conduct a comprehensive meta-analysis of AI safety benchmarks, empirically analyzing their correlation with general capabilities across dozens of models and providing a survey of existing directions in AI safety. Our findings reveal that many safety benchmarks highly correlate with both upstream model capabilities and training compute, potentially enabling "safetywashing"--where capability improvements are misrepresented as safety advancements. Based on these findings, we propose an empirical foundation for developing more meaningful safety metrics and define AI safety in a machine learning research context as a set of clearly delineated research goals that are empirically separable from generic capabilities advancements. In doing so, we aim to provide a more rigorous framework for AI safety research, advancing the science of safety evaluations and clarifying the path towards measurable progress.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Larger and newer models still showed stigma and failed to respond appropriately.
“We found larger and newer models (with, in theory, better safety filtering and tuning [114, 157]) still showed stigma (Fig. 1 and 6) and failed to respond appropriately (Fig. 4).”
From Expressing stigma and inappropriate responses prevents LLMs from safely replacing mental health providers. (2025) · cites this paper as [114]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 74,248 · see citing works
RoBERTa: A Robustly Optimized BERT Pretraining Approach
Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., & Stoyanov, V. (2019) · arXiv · doi:10.48550/arxiv.1907.11692 · Cited by 16,515 · see citing works
Learning Transferable Visual Models From Natural Language Supervision
Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al. (2021) · arXiv · doi:10.48550/arxiv.2103.00020 · Cited by 15,293 · see citing works
Training language models to follow instructions with human feedback
Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C. L., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., et al. (2022) · arXiv · doi:10.48550/arxiv.2203.02155 · Cited by 4,208 · see citing works
Llama 2: Open Foundation and Fine-Tuned Chat Models
Touvron, H., Martin, L., Stone, K., Albert, P., Almahairi, A., Babaei, Y., Bashlykov, N., Batra, S., Bhargava, P., Bhosale, S., et al. (2023) · arXiv · doi:10.48550/arxiv.2307.09288 · Cited by 2,604 · see citing works
Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena
Zheng, L., Chiang, W.-L., Sheng, Y., Zhuang, S., Wu, Z., Zhuang, Y., Lin, Z., Li, Z., Li, D., Xing, E. P., et al. (2023) · arXiv · doi:10.48550/arxiv.2306.05685 · Cited by 1,888 · see citing works
Scaling Laws for Neural Language Models
Kaplan, J., McCandlish, S., Henighan, T., Brown, T. B., Chess, B., Child, R., Gray, S., Radford, A., Wu, J., & Amodei, D. (2020) · arXiv · Cited by 1,749 · cited 5× · see citing works
“Recent advances in observational scaling laws have provided a methodology that allows researchers to gain a deeper understanding of the underlying capabilities of machine learning models [41, 55].”
InfluentialConcrete Problems in AI Safety
Amodei, D., Olah, C., Steinhardt, J., Christiano, P., Schulman, J., & Mané, D. (2016) · arXiv · doi:10.48550/arxiv.1606.06565 · Cited by 1,314 · see citing works
Emergent Abilities of Large Language Models
Wei, J., Tay, Y., Bommasani, R., Raffel, C., Zoph, B., Borgeaud, S., Yogatama, D., Bosma, M., Zhou, D., Metzler, D., et al. (2022) · arXiv · doi:10.48550/arxiv.2206.07682 · Cited by 1,264 · see citing works
Tree of Thoughts: Deliberate Problem Solving with Large Language Models
Yao, S., Yu, D., Zhao, J., Shafran, I., Griffiths, T. L., Cao, Y., & Narasimhan, K. (2023) · arXiv · doi:10.48550/arxiv.2305.10601 · Cited by 1,158 · see citing works
DINOv2: Learning Robust Visual Features without Supervision
Oquab, M., Darcet, T., Moutakanni, T., Vo, H., Szafraniec, M., Khalidov, V., Fernandez, P., Haziza, D., Massa, F., El-Nouby, A., et al. (2023) · arXiv · doi:10.48550/arxiv.2304.07193 · Cited by 1,078 · see citing works
AugMix: A Simple Data Processing Method to Improve Robustness and UncertaintyPreprint
Hendrycks, D., Mu, N., Cubuk, E. D., Zoph, B., Gilmer, J., & Lakshminarayanan, B. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1912.02781 · Cited by 893 · see citing works
Self-citationThe Falcon Series of Open Language ModelsPreprint
Almazrouei, E., Alobeidli, H., AlShamsi, A., Cappelli, A., Cojocaru, R.-A., Debbah, M., Goffinet, É., Hesslow, D., Launay, J., Malartic, Q., et al. (2023) · arXiv.org · doi:10.48550/arxiv.2311.16867 · Cited by 813 · see citing works
Training Verifiers to Solve Math Word Problems
Cobbe, K., Kosaraju, V., Bavarian, M., Chen, M., Jun, H., Kaiser, L., Plappert, M., Tworek, J., Hilton, J., Nakano, R., et al. (2021) · arXiv · doi:10.48550/arxiv.2110.14168 · Cited by 575 · see citing works
Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge
Clark, P., Cowhey, I., Etzioni, O., Khot, T., Sabharwal, A., Schoenick, C., & Tafjord, O. (2018) · arXiv · doi:10.48550/arxiv.1803.05457 · Cited by 462 · see citing works
Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback
Bai, Y., Jones, A., Ndousse, K., Askell, A., Chen, A., DasSarma, N., Drain, D., Fort, S., Ganguli, D., Henighan, T., et al. (2022) · arXiv · doi:10.48550/arxiv.2204.05862 · Cited by 442 · see citing works
Deep Learning Scaling is Predictable, Empirically
Hestness, J., Narang, S., Ardalani, N., Diamos, G., Jun, H., Kianinejad, H., Patwary, M. M. A., Yang, Y., & Zhou, Y. (2017) · arXiv · doi:10.48550/arxiv.1712.00409 · Cited by 427 · see citing works
Jiang, A. Q., Sablayrolles, A., Mensch, A., Bamford, C., Chaplot, D. S., Casas, D. D. L., Bressand, F., Lengyel, G., Lample, G., Saulnier, L., et al. (2023) · arXiv · doi:10.48550/arxiv.2310.06825 · Cited by 340 · see citing works
HaluEval: A Large-Scale Hallucination Evaluation Benchmark for Large Language Models
Li, J., Cheng, X., Zhao, W. X., Nie, J.-Y., & Wen, J.-R. (2023) · arXiv · doi:10.48550/arxiv.2305.11747 · Cited by 340 · see citing works
Bai, J., Bai, S., Chu, Y., Cui, Z., Dang, K., Deng, X., Fan, Y., Ge, W., Han, Y., Huang, F., et al. (2023) · arXiv · doi:10.48550/arxiv.2309.16609 · Cited by 332 · see citing works
Evaluating the Moral Beliefs Encoded in LLMs
(2023) · doi:10.48550/arxiv.2307.14324 · Cited by 315
Gemma: Open Models Based on Gemini Research and Technology
Gemma Team, Mesnard, T., Hardin, C., Dadashi, R., Bhupatiraju, S., Pathak, S., Sifre, L., Rivière, M., Kale, M. S., Love, J., et al. (2024) · arXiv · doi:10.48550/arxiv.2403.08295 · Cited by 284 · see citing works
Holistic Evaluation of Language Models
Liang, P., Bommasani, R., Lee, T., Tsipras, D., Soylu, D., Yasunaga, M., Zhang, Y., Narayanan, D., Wu, Y., Kumar, A., et al. (2022) · arXiv · doi:10.48550/arxiv.2211.09110 · Cited by 255 · see citing works
Are Emergent Abilities of Large Language Models a Mirage?
Schaeffer, R., Miranda, B., & Koyejo, S. (2023) · arXiv · doi:10.48550/arxiv.2304.15004 · Cited by 200 · see citing works
Inference-Time Intervention: Eliciting Truthful Answers from a Language Model
Li, K., Patel, O., Viégas, F., Pfister, H., & Wattenberg, M. (2023) · arXiv · doi:10.48550/arxiv.2306.03341 · Cited by 184 · see citing works
Tree of Attacks: Jailbreaking Black-Box LLMs Automatically
Mehrotra, A., Zampetakis, M., Kassianik, P., Nelson, B., Anderson, H., Singer, Y., & Karbasi, A. (2023) · arXiv · doi:10.48550/arxiv.2312.02119 · Cited by 183 · see citing works
Jiang, A. Q., Sablayrolles, A., Roux, A., Mensch, A., Savary, B., Bamford, C., Chaplot, D. S., Casas, D. D. L., Hanna, E. B., Bressand, F., et al. (2024) · arXiv · doi:10.48550/arxiv.2401.04088 · Cited by 143 · see citing works
Self-supervised Pretraining of Visual Features in the Wild
Goyal, P., Caron, M., Lefaudeux, B., Xu, M., Wang, P., Pai, V., Singh, M., Liptchinsky, V., Misra, I., Joulin, A., et al. (2021) · arXiv · doi:10.48550/arxiv.2103.01988 · Cited by 141 · see citing works
Inverse scaling can become U-shapedPreprint
Wei, J., Kim, N., Tay, Y., & Le, Q. V. (2022) · Conference on Empirical Methods in Natural Language Processing · doi:10.48550/arxiv.2211.02011 · Cited by 99 · see citing works
DeepSeek LLM: Scaling Open-Source Language Models with Longtermism
DeepSeek-AI, :, Bi, X., Chen, D., Chen, G., Chen, S., Dai, D., Deng, C., Ding, H., Dong, K., et al. (2024) · arXiv · doi:10.48550/arxiv.2401.02954 · Cited by 99 · see citing works
Unsolved Problems in ML Safety
Hendrycks, D., Carlini, N., Schulman, J., & Steinhardt, J. (2021) · arXiv · doi:10.48550/arxiv.2109.13916 · Cited by 95 · see citing works
Self-citationThe WMDP Benchmark: Measuring and Reducing Malicious Use With Unlearning
Li, N., Pan, A., Gopal, A., Yue, S., Berrios, D., Gatti, A., Li, J. D., Dombrowski, A.-K., Goel, S., Phan, L., et al. (2024) · arXiv · doi:10.48550/arxiv.2403.03218 · Cited by 90 · see citing works
Self-citationX-Risk Analysis for AI ResearchPreprint
Hendrycks, D., & Mazeika, M. (2022) · arXiv.org · doi:10.48550/arxiv.2206.05862 · Cited by 88 · see citing works
Self-citationRLAIF vs. RLHF: Scaling Reinforcement Learning from Human Feedback with AI Feedback
Lee, H., Phatale, S., Mansoor, H., Mesnard, T., Ferret, J., Lu, K., Bishop, C., Hall, E., Carbune, V., Rastogi, A., et al. (2023) · arXiv · doi:10.48550/arxiv.2309.00267 · Cited by 87 · cited 1× · see citing works
“It also features 2,048 GB of RAM, 24 TB of NVMe storage, and offers a network throughput of 1.6 Tbps.”
InfluentialTraining Trajectories of Language Models Across ScalesPreprint
Xia, M., Artetxe, M., Zhou, C., et al. (2022) · Annual Meeting of the Association for Computational Linguistics · doi:10.48550/arxiv.2212.09803 · Cited by 86 · see citing works
WinoGrande: An Adversarial Winograd Schema Challenge at ScalePreprint
Sakaguchi, K., Bras, R. J. L., Bhagavatula, C., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1907.10641 · Cited by 84 · see citing works
ConvNeXt V2: Co-designing and Scaling ConvNets with Masked AutoencodersPreprint
Woo, S., Debnath, S., Hu, R., et al. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2301.00808 · Cited by 66 · see citing works
Can LLMs Follow Simple Rules?Preprint
Mu, N., Chen, S., Wang, Z., et al. (2023) · arXiv.org · doi:10.48550/arxiv.2311.04235 · Cited by 57 · see citing works
Discovering Language Model Behaviors with Model-Written Evaluations
Perez, E., Ringer, S., Lukošiūtė, K., Nguyen, K., Chen, E., Heiner, S., Pettit, C., Olsson, C., Kundu, S., Kadavath, S., et al. (2022) · arXiv · doi:10.48550/arxiv.2212.09251 · Cited by 53 · see citing works
Yi: Open Foundation Models by 01.AI
AI, 0., :, Young, A., Chen, B., Li, C., Huang, C., Zhang, G., Zhang, G., Wang, G., Li, H., et al. (2024) · arXiv · doi:10.48550/arxiv.2403.04652 · Cited by 50 · see citing works
Shen, X., Chen, Z., Backes, M., et al. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2308.03825 · Cited by 42 · see citing works
Truthful AI: Developing and governing AI that does not lie
Evans, O., Cotton-Barratt, O., Finnveden, L., Bales, A., Balwit, A., Wills, P., Righetti, L., & Saunders, W. (2021) · arXiv · doi:10.48550/arxiv.2110.06674 · Cited by 39 · see citing works
Instruction-Following Evaluation for Large Language Models
Zhou, J., Lu, T., Mishra, S., Brahma, S., Basu, S., Luan, Y., Zhou, D., & Hou, L. (2023) · arXiv · doi:10.48550/arxiv.2311.07911 · Cited by 39 · see citing works
The shutdown problem: an AI engineering puzzle for decision theorists
Thornley, E. (2024) · Philosophical Studies · doi:10.1007/s11098-024-02153-3 · Cited by 34 · see citing works
Measuring Progress on Scalable Oversight for Large Language Models
Bowman, S. R., Hyun, J., Perez, E., Chen, E., Pettit, C., Heiner, S., Lukošiūtė, K., Askell, A., Jones, A., Chen, A., et al. (2022) · arXiv · doi:10.48550/arxiv.2211.03540 · Cited by 34 · see citing works
GPQA: A Graduate-Level Google-Proof Q&A Benchmark
Rein, D., Hou, B. L., Stickland, A. C., Petty, J., Pang, R. Y., Dirani, J., Michael, J., & Bowman, S. R. (2023) · arXiv · doi:10.48550/arxiv.2311.12022 · Cited by 30 · see citing works
Inverse Scaling: When Bigger Isn't Better
McKenzie, I. R., Lyzhov, A., Pieler, M., Parrish, A., Mueller, A., Prabhu, A., McLean, E., Kirtland, A., Ross, A., Liu, A., et al. (2023) · arXiv · doi:10.48550/arxiv.2306.09479 · Cited by 29 · see citing works
HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal
Mazeika, M., Phan, L., Yin, X., Zou, A., Wang, Z., Mu, N., Sakhaee, E., Li, N., Basart, S., Li, B., et al. (2024) · arXiv · doi:10.48550/arxiv.2402.04249 · Cited by 28
Self-citationScaling Laws for Neural Machine TranslationPreprint
Ghorbani, B., Fırat, O., Freitag, M., Bapna, A., Krikun, M., Garcia, X., Chelba, C. I., & Cherry, C. A. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2109.07740 · Cited by 23 · see citing works
OpenOOD v1.5: Enhanced Benchmark for Out-of-Distribution DetectionPreprint
Zhang, J., Yang, J., Wang, P., et al. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2306.09301 · Cited by 20 · see citing works
The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution GeneralizationPreprint
Hendrycks, D., Basart, S., Mu, N., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.16241 · Cited by 19 · see citing works
Self-citationGoal Misgeneralization: Why Correct Specifications Aren't Enough For Correct Goals
Shah, R., Varma, V., Kumar, R., Phuong, M., Krakovna, V., Uesato, J., & Kenton, Z. (2022) · arXiv · doi:10.48550/arxiv.2210.01790 · Cited by 16 · see citing works
Hidden Incentives for Auto-Induced Distributional Shift
Krueger, D., Maharaj, T., & Leike, J. (2020) · arXiv · doi:10.48550/arxiv.2009.09153 · Cited by 14 · see citing works
CyberSecEval 2: A Wide-Ranging Cybersecurity Evaluation Suite for Large Language Models
Bhatt, M., Chennabasappa, S., Li, Y., Nikolaidis, C., Song, D., Wan, S., Ahmad, F., Aschermann, C., Chen, Y., Kapil, D., et al. (2024) · arXiv · doi:10.48550/arxiv.2404.13161 · Cited by 12 · see citing works
Evaluating and Mitigating Discrimination in Language Model Decisions
Tamkin, A., Askell, A., Lovitt, L., Durmus, E., Joseph, N., Kravec, S., Nguyen, K., Kaplan, J., & Ganguli, D. (2023) · arXiv · doi:10.48550/arxiv.2312.03689 · Cited by 11 · cited 2× · see citing works
“This correlation means that simply enhancing the upstream general capabilities of models, such as by scaling parameters and increasing training data, often boosts performance across all benchmarks indiscriminately [41, 52].”
QuALITY: Question Answering with Long Input Texts, Yes!
Pang, R. Y., Parrish, A., Joshi, N., Nangia, N., Phang, J., Chen, A., Padmakumar, V., Ma, J., Thompson, J., He, H., et al. (2021) · arXiv · doi:10.48550/arxiv.2112.08608 · Cited by 10 · see citing works
Forecasting Future World Events with Neural NetworksPreprint
Zou, A., Xiao, T., Jia, R., et al. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2206.15474 · Cited by 6 · see citing works
LogiQA: A Challenge Dataset for Machine Reading Comprehension with Logical ReasoningPreprint
Jian, L., Cui, L., Liu, H., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.08124 · Cited by 5 · see citing works
Compression Represents Intelligence Linearly
Huang, Y., Zhang, J., Shan, Z., & He, J. (2024) · arXiv · doi:10.48550/arxiv.2404.09937 · Cited by 5 · see citing works
Understanding Emergent Abilities of Language Models from the Loss Perspective
Du, Z., Zeng, A., Dong, Y., & Tang, J. (2024) · arXiv · doi:10.48550/arxiv.2403.15796 · Cited by 5 · see citing works
Observational Scaling Laws and the Predictability of Language Model Performance
Ruan, Y., Maddison, C. J., & Hashimoto, T. (2024) · arXiv · doi:10.48550/arxiv.2405.10938 · Cited by 3 · see citing works
Math-Shepherd: Verify and Reinforce LLMs Step-by-step without Human Annotations
Wang, P., Li, L., Shao, Z., Xu, R. X., Dai, D., Li, Y., Chen, D., Wu, Y., & Sui, Z. (2023) · arXiv · doi:10.48550/arxiv.2312.08935 · Cited by 3 · see citing works
Why Has Predicting Downstream Capabilities of Frontier AI Models with Scale Remained Elusive?
Schaeffer, R., Schoelkopf, H., Miranda, B., Mukobi, G., Madan, V., Ibrahim, A., Bradley, H., Biderman, S., & Koyejo, S. (2024) · arXiv · doi:10.48550/arxiv.2406.04391 · Cited by 2 · see citing works
Self-citationBERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805
Flagged first, then by citations
Research works printed but not linked: 59
- Ethan Perez, Sam Ringer, Kamilė Lukošiūtė, Karina Nguyen, Edwin Chen, Scott Heiner, Craig Pettit, Catherine Olsson, Sandipan Kundu, Saurav Kadavath, et al. Discovering language model behaviors with model-written evaluations. In ACL, 2023.
- Stephanie Lin, Jacob Hilton, and Owain Evans. Truthfulqa: Measuring how models mimic human falsehoods. In ACL, 2022.
- Dan Hendrycks, Kevin Zhao, Steven Basart, Jacob Steinhardt, and Dawn Song. Natural adversarial examples. In CVPR, 2021.
- Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B. Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. Scaling laws for neural language models. arXiv preprint arXiv:2001.08361, 2020. · arXiv
- Joel Hestness, Sharan Narang, Newsha Ardalani, Gregory Diamos, Heewoo Jun, Hassan Kianinejad, Md Mostofa Ali Patwary, Yang Yang, and Yanqi Zhou. Deep learning scaling is predictable, empirically. arXiv preprint arXiv:1712.00409, 2017. · arXiv
- Niklas Muennighoff, Alexander Rush, Boaz Barak, Teven Le Scao, Nouamane Tazi, Alek- sandra Piktus, Sampo Pyysalo, Thomas Wolf, and Colin A Raffel. Scaling data-constrained language models. NeurIPS, 2024.
- Jordan Hoffmann, Sebastian Borgeaud, Arthur Mensch, Elena Buchatskaya, Trevor Cai, Eliza Rutherford, Diego de Las Casas, Lisa Anne Hendricks, Johannes Welbl, Aidan Clark, et al. Training compute-optimal large language models. In NeurIPS, 2024.
- Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In CVPR, 2016.
- Xiaohua Zhai, Alexander Kolesnikov, Neil Houlsby, and Lucas Beyer. Scaling vision trans- formers. In CVPR, 2022.
- Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, and Ross Girshick. Masked autoencoders are scalable vision learners. In CVPR, 2022.
- William Peebles and Saining Xie. Scalable diffusion models with transformers. In ICCV, 2023.
- David Ilić. Unveiling the general intelligence factor in language models: A psychometric approach. arXiv preprint arXiv:2310.11616, 2023. · arXiv
- Pablo Villalobos. Scaling laws literature review. Epoch AI, 2023.
- Kaiming He, Ross Girshick, and Piotr Dollar. Rethinking imagenet pre-training. In ICCV, 2019.
- Simon Kornblith, Jonathon Shlens, and Quoc V Le. Do better imagenet models transfer better? In CVPR, 2019.
- Nick Bostrom. Existential risks: Analyzing human extinction scenarios and related hazards. Journal of Evolution and Technology, 2002.
- Dan Hendrycks, Andy Zou, Mantas Mazeika, Leonard Tang, Bo Li, Dawn Song, and Jacob Steinhardt. Pixmix: Dreamlike pictures comprehensively improve safety measures. CVPR, 2022.
- Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Measuring massive multitask language understanding. ICLR, 2021.
- Yonatan Bisk, Rowan Zellers, Ronan Le Bras, Jianfeng Gao, and Yejin Choi. Piqa: Reasoning about physical commonsense in natural language. In AAAI, 2020.
- Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi. Hellaswag: Can a machine really finish your sentence? In ACL, 2019.
- Melissa Roemmele, Cosmin Bejan, and Andrew Gordon. Choice of plausible alternatives: An evaluation of commonsense causal reasoning. In AAAI Spring Symposium, 2011.
- Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. Measuring mathematical problem solving with the math dataset. NeurIPS, 2021.
- Denis Paperno, Germán Kruszewski, Angeliki Lazaridou, Ngoc Quan Pham, Raffaella Bernardi, Sandro Pezzelle, Marco Baroni, Gemma Boleda, and Raquel Fernandez. The lambada dataset: Word prediction requiring a broad discourse context. In ACL, 2016.
- BIG bench authors. Beyond the imitation game: Quantifying and extrapolating the capabilities of language models. TMLR, 2023.
- John Rawls. A Theory of Justice. Belknap Press, United States, 1971. ISBN 978-0-674- 00078-0.
- Dan Hendrycks. Introduction to AI Safety, Ethics and Society. Taylor & Francis, 2025.
- Dan Hendrycks, Collin Burns, Steven Basart, Andrew Critch, Jerry Li, Dawn Song, and Jacob Steinhardt. Aligning ai with shared human values. ICLR, 2021.
- Alexander Pan, Jun Shern Chan, Andy Zou, Nathaniel Li, Steven Basart, Thomas Woodside, Hanlin Zhang, Scott Emmons, and Dan Hendrycks. Do the rewards justify the means? measuring trade-offs between rewards and ethical behavior in the machiavelli benchmark. In ICML, 2023.
- Peter Railton. Ethics and artificial intelligence. Oxford Uehiro Centre for Practical Ethics Lecture Series, University of Oxford, 2022.
- Gilbert Harman. Moral relativism. In Gilbert Harman and Judith Jarvis Thompson, editors, Moral Relativism and Moral Objectivity, pages 3-64. Blackwell Publishers, Cambridge, MA, 1996.
- Alicia Parrish, Angelica Chen, Nikita Nangia, Vishakh Padmakumar, Jason Phang, Jana Thompson, Phu Mon Htut, and Samuel Bowman. Bbq: A hand-built bias benchmark for question answering. In ACL, 2022.
- Nikita Nangia, Clara Vania, Rasika Bhalerao, and Samuel R. Bowman. Crows-pairs: A challenge dataset for measuring social biases in masked language models. In EMNLP, 2020.
- Matthew D. Adler. Measuring Social Welfare: An Introduction. Oxford University Press, 2019. ISBN 9780190643065. doi: 10.1093/oso/9780190643027.001.0001. · DOI
- Rachel Rudinger, Jason Naradowsky, Brian Leonard, and Benjamin Van Durme. Gender bias in coreference resolution. In NAACL, 2018.
- Vedant Shah, Anirudh Goyal, Dingli Yu, Kaifeng Lyu, Simon Park, Nan Rosemary Ke, James Lloyd McClelland, Yoshua Bengio, Sanjeev Arora, and Michael Curtis Mozer. Ai- assisted generation of difficult math questions. In AI for Math Workshop @ ICML, 2024.
- Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q Weinberger. On calibration of modern neural networks. In ICML, 2017.
- Dan Hendrycks, Kimin Lee, and Mantas Mazeika. Using pre-training can improve model robustness and uncertainty. In ICML, 2019.
- Yixin Nie, Adina Williams, Emily Dinan, Mohit Bansal, Jason Weston, and Douwe Kiela. Adversarial nli: A new benchmark for natural language understanding. In ACL, 2020.
- Boxin Wang, Chejian Xu, Shuohang Wang, Zhe Gan, Yu Cheng, Jianfeng Gao, Ahmed Hassan Awadallah, and Bo Li. Adversarial glue: A multi-task benchmark for robustness evaluation of language models. In NeurIPS, 2021.
- Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel Bowman. Glue: A multi-task benchmark and analysis platform for natural language understanding. In EMNLP, 2018.
- Boxin Wang, Weixin Chen, Hengzhi Pei, Chulin Xie, Mintong Kang, Chenhui Zhang, Chejian Xu, Zidi Xiong, Ritik Dutta, Rylan Schaeffer, et al. Decodingtrust: A comprehensive assess- ment of trustworthiness in gpt models. NeurIPS, 2023.
- Andy Zou, Zifan Wang, J Zico Kolter, and Matt Fredrikson. Universal and transferable adversarial attacks on aligned language models. arXiv preprint arXiv:2307.15043, 2023. · arXiv
- Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. In ICLR, 2018.
- Nate Soares, Benja Fallenstein, Eliezer Yudkowsky, and Stuart Armstrong. Corrigibility. In AAAI Publications. Association for the Advancement of Artificial Intelligence, 2015.
- Dan Hendrycks and Thomas Dietterich. Benchmarking neural network robustness to common corruptions and perturbations. ICLR, 2019.
- Hadi Salman, Andrew Ilyas, Logan Engstrom, Ashish Kapoor, and Aleksander Madry. Do adversarially robust imagenet models transfer better? NeurIPS, 2020.
- Zhuang Liu, Hanzi Mao, Chao-Yuan Wu, Christoph Feichtenhofer, Trevor Darrell, and Saining Xie. A convnet for the 2020s. In CVPR, 2022.
- Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convolutional neural networks. Advances in neural information processing systems, 25, 2012.
- Sergey Zagoruyko and Nikos Komodakis. Wide residual networks, 2017. URL https: //arxiv.org/abs/1605.07146.
- Saining Xie, Ross Girshick, Piotr Dollár, Zhuowen Tu, and Kaiming He. Aggregated residual transformations for deep neural networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1492-1500, 2017.
- Gao Huang, Zhuang Liu, Laurens Van Der Maaten, and Kilian Q Weinberger. Densely connected convolutional networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 4700-4708, 2017.
- Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hierarchical vision transformer using shifted windows. In ICCV, 2021.
- Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. In International Conference on Learning Representations, 2020.
- Andreas Steiner, Alexander Kolesnikov, Xiaohua Zhai, Ross Wightman, Jakob Uszkoreit, and Lucas Beyer. How to train your vit? data, augmentation, and regularization in vision transformers. TMLR, 2022.
- Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In ICML, 2021.
- Karttikeya Mangalam, Haoqi Fan, Yanghao Li, Chao-Yuan Wu, Bo Xiong, Christoph Feicht- enhofer, and Jitendra Malik. Reversible vision transformers. In CVPR, 2022.
- Zhexin Zhang, Leqi Lei, Lindong Wu, Rui Sun, Yongkang Huang, Chong Long, Xiao Liu, Xuanyu Lei, Jie Tang, and Minlie Huang. Safetybench: Evaluating the safety of large language models with multiple choice questions. arXiv preprint arXiv:2309.07045, 2023. · arXiv
- Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhari- wal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. NeurIPS, 2020.
- Thomas Hartvigsen, Saadia Gabriel, Hamid Palangi, Maarten Sap, Dipankar Ray, and Ece Kamar. Toxigen: A large-scale machine-generated dataset for implicit and adversarial hate speech detection. ACL, 2022.
Cited by
Sorted by citations
Moore, J., Grabb, D., Agnew, W., Klyman, K., Chancellor, S., Ong, D. C., & Haber, N. (2025) · Proceedings of the 2025 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3715275.3732039 · Cited by 204 · see citing works
- Authors
- Ren, R., Basart, S., Khoja, A., Gatti, A., Phan, L., Yin, X., Mazeika, M., Pan, A., Mukobi, G., Kim, R. H., Fitz, S., & Hendrycks, D.
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2024
- Pages
- 68559-68594
- ISSN
- 1049-5258 (ISSN-L)
- arXiv
- 2407.21792
- OpenAlex
- W4415797159
- Cited by
- 10
- References
- 64
- Citations by year
- 2026: 6 · 2025: 3 · 2024: 1
- FWCI
- 4.54
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- neural information processing systems (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Social Sciences › Safety Research › Ethics and Social Impacts of AI
- Physical Sciences › Engineering › Safety, Risk, Reliability and Quality › Safety Systems Engineering in Autonomy
- Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
- Keywords
- benchmarking, AI safety, evaluation metrics
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
