CoChat
Sign Up
Cited by 86

The DOI matches its Crossref record; 50 of the 50 references it cites were found in a registry or an index.

Black-Box Access is Insufficient for Rigorous AI Audits

Casper, Stephen, Ezell, Carson, Siegmann, Charlotte, Kolt, Noam, Curtis, Taylor Lynn, Bucknall, Benjamin, Haupt, Andreas, Wei, Kevin, Scheurer, Jérémy, Hobbhahn, Marius, Sharkey, Lee, Krishna, Satyapriya, Von Hagen, Marvin, Alberti, Silas, Chan, Alan, Sun, Qinyi, Gerovitch, Michael, Bau, David, Tegmark, Max, Krueger, David, Hadfield-Menell, Dylan·2024·The 2024 ACM Conference on Fairness, Accountability, and Transparency
DOI 10.1145/3630106.3659037
Open this paper in CoChat — free account

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1145/3630106.3659037

Cites

50 of 50 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • An understanding of model internals can help AI labs assess the risks from frontier models.

    “An understanding of model internals can help AI labs assess the risks from frontier models (Chang et al., 2024;Shevlane et al., 2023;Casper et al., 2024) and thus better fulfill their obligations under the EU AI Act to”

    From Open Problems in Mechanistic Interpretability (2025) · cites this paper as Casper et al., 2024

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Flagged first, then by citations

Research works printed but not linked: 261

  1. 10.1145/3461702.3462563 · DOI
  2. Abubakar Abid, Mert Yuksekgonul, and James Zou. 2022. Meaningfully debugging model mistakes using conceptual counterfactual explanations. In International Conference on Machine Learning. PMLR, 66–88.
  3. Julius Adebayo, Michael Muelly, Ilaria Liccardi, and Been Kim. 2020. Debugging tests for model explanations. arXiv preprint arXiv:2011.05429 (2020). · arXiv
  4. Chirag Agarwal, Satyapriya Krishna, Eshika Saxena, Martin Pawelczyk, Nari Johnson, Isha Puri, Marinka Zitnik, and Himabindu Lakkaraju. 2022. Openxai: Towards a transparent evaluation of model explanations. Advances in Neural Information Processing Systems 35 (2022), 15784–15799.
  5. Ulrich Aivodji, Hiromi Arai, Olivier Fortineau, Sébastien Gambs, Satoshi Hara, and Alain Tapp. 2019. Fairwashing: the risk of rationalization. In Proceedings of the 36th International Conference on Machine Learning. PMLR, 161–170. https://proceedings.mlr.press/v97/aivodji19a.html ISSN: 2640-3498.
  6. 10.1109/ACCESS.2018.2807385 · DOI · Paper page
  7. Guillaume Alain and Yoshua Bengio. 2018. Understanding intermediate layers using linear classifier probes. (2018). arxiv:1610.01644 [stat.ML] · arXiv
  8. Alex Albert. 2023. Jailbreak Chat. (2023). https://www.jailbreakchat.com/
  9. Markus Anderljung, Joslyn Barnhart, Jade Leung, Anton Korinek, Cullen O’Keefe, Jess Whittlestone, Shahar Avin, Miles Brundage, Justin Bullock, Duncan Cass-Beggs, 2023. Frontier AI regulation: Managing emerging risks to public safety. arXiv preprint arXiv:2307.03718 (2023). · arXiv
  10. Markus Anderljung Everett Thornton Smith Joe O’Brien Lisa Soder Benjamin Bucknall Emma Bluemke Jonas Schuett Robert Trager Lacey Strahm and Rumman Chowdhury. 2023. Towards Publicly Accountable Frontier LLMs: Building an External Scrutiny Ecosystem under the ASPIRE Framework. (2023). arxiv:2311.14711 [cs.CY] · arXiv
  11. Julia Angwin, Jeff Larson, Surya Mattu, and Lauren Kirchner. 2022. Machine bias. In Ethics of data and analytics. Auerbach Publications, 254–264.
  12. Omer Antverg and Yonatan Belinkov. 2021. On the pitfalls of analyzing individual neurons in language models. arXiv preprint arXiv:2110.07483 (2021). · arXiv
  13. Ben Athiwaratkun and Keegan Kang. 2015. Feature representation in convolutional neural networks. arXiv preprint arXiv:1507.02313 (2015). · arXiv
  14. 10.1016/j.jfineco.2019.06.001 · DOI
  15. 10.1162/coli_a_00422 · DOI
  16. Yoshua Bengio, Geoffrey Hinton, Andrew Yao, Dawn Song, Pieter Abbeel, Yuval Noah Harari, Ya-Qin Zhang, Lan Xue, Shai Shalev-Shwartz, Gillian Hadfield, 2023. Managing AI Risks in an Era of Rapid Progress. arXiv preprint arXiv:2310.17688 (2023). · arXiv
  17. Siddhant Bhambri, Sumanyu Muku, Avinash Tulasi, and Arun Balaji Buduru. 2019. A survey of black-box adversarial attacks on computer vision models. arXiv preprint arXiv:1912.01667 (2019). · arXiv
  18. 10.1145/3531146.3533083 · DOI · Paper page
  19. Abeba Birhane, Vinay Prabhu, Sang Han, Vishnu Naresh Boddeti, and Alexandra Sasha Luccioni. 2023. Into the LAIONs Den: Investigating Hate in Multimodal Datasets. arXiv preprint arXiv:2311.03449 (2023). · arXiv
  20. Abeba Birhane, Vinay Uday Prabhu, and Emmanuel Kahembwe. 2021. Multimodal datasets: misogyny, pornography, and malignant stereotypes. arXiv preprint arXiv:2110.01963 (2021). · arXiv
  21. Abeba Birhane Ryan Steed Victor Ojewale Briana Vecchione and Inioluwa Deborah Raji. 2024. AI auditing: The Broken Bus on the Road to AI Accountability. arxiv:2401.14462 [cs.CY] · DOI · Paper page
  22. Emma Bluemke, Tantum Collins, Ben Garfinkel, and Andrew Trask. 2023. Exploring the Relevance of Data Privacy-Enhancing Technologies for AI Governance Use Cases. (March 2023). https://arxiv.org/abs/2303.08956v2 · arXiv
  23. 10.1111/j.1540-6261.2011.01708.x · DOI
  24. Tolga Bolukbasi, Kai-Wei Chang, James Y Zou, Venkatesh Saligrama, and Adam T Kalai. 2016. Man is to computer programmer as woman is to homemaker? debiasing word embeddings. Advances in neural information processing systems 29 (2016).
  25. Rishi Bommasani, Kevin Klyman, Shayne Longpre, Sayash Kapoor, Nestor Maslej, Betty Xiong, Daniel Zhang, and Percy Liang. 2023. The Foundation Model Transparency Index. (Oct. 2023). http://arxiv.org/abs/2310.12941 arXiv:2310.12941 [cs]. · arXiv
  26. Andres M Bran, Sam Cox, Andrew D White, and Philippe Schwaller. 2023. ChemCrow: Augmenting large-language models with chemistry tools. arXiv preprint arXiv:2304.05376 (2023). · arXiv
  27. Trenton Bricken, Adly Templeton, Joshua Batson, Brian Chen, Adam Jermyn, Tom Conerly, Nick Turner, Cem Anil, Carson Denison, Amanda Askell, Robert Lasenby, Yifan Wu, Shauna Kravec, Nicholas Schiefer, Tim Maxwell, Nicholas Joseph, Zac Hatfield-Dodds, Alex Tamkin, Karina Nguyen, Brayden McLean, Josiah E Burke, Tristan Hume, Shan Carter, Tom Henighan, and Christopher Olah. 2023. Towards Monosemanticity: Decomposing Language Models With Dictionary Learning. Transformer Circuits Thread (2023). https://transformer-circuits.pub/2023/monosemantic-features/index.html.
  28. 10.2139/ssrn.3817520 · DOI · Paper page
  29. 10.1109/REW57809.2023.00062 · DOI
  30. Joy Buolamwini and Timnit Gebru. 2018. Gender shades: Intersectional accuracy disparities in commercial gender classification. In Conference on fairness, accountability and transparency. PMLR, 77–91.
  31. Collin Burns, Haotian Ye, Dan Klein, and Jacob Steinhardt. 2022. Discovering latent knowledge in language models without supervision. arXiv preprint arXiv:2212.03827 (2022). · arXiv
  32. 10.23915/distill.00024 · DOI
  33. Nicholas Carlini, Daphne Ippolito, Matthew Jagielski, Katherine Lee, Florian Tramer, and Chiyuan Zhang. 2022. Quantifying memorization across neural language models. arXiv preprint arXiv:2202.07646 (2022). · arXiv
  34. Nicholas Carlini, Matthew Jagielski, Christopher A Choquette-Choo, Daniel Paleka, Will Pearce, Hyrum Anderson, Andreas Terzis, Kurt Thomas, and Florian Tramèr. 2023. Poisoning web-scale training datasets is practical. arXiv preprint arXiv:2302.10149 (2023). · arXiv
  35. Nicholas Carlini, Milad Nasr, Christopher A Choquette-Choo, Matthew Jagielski, Irena Gao, Anas Awadalla, Pang Wei Koh, Daphne Ippolito, Katherine Lee, Florian Tramer, 2023. Are aligned neural networks adversarially aligned?arXiv preprint arXiv:2306.15447 (2023). · arXiv
  36. Nicholas Carlini, Florian Tramer, Eric Wallace, Matthew Jagielski, Ariel Herbert-Voss, Katherine Lee, Adam Roberts, Tom Brown, Dawn Song, Ulfar Erlingsson, 2021. Extracting training data from large language models. In 30th USENIX Security Symposium (USENIX Security 21). 2633–2650.
  37. Shan Carter Zan Armstrong Ludwig Schubert Ian Johnson and Chris Olah. 2019. Exploring neural networks with activation atlases. Distill. (2019). · DOI · Paper page
  38. Stephen Casper, Zifan Guo, Shreya Mogulothu, Zachary Marinov, Chinmay Deshpande, Rui-Jie Yew, Zheng Dai, and Dylan Hadfield-Menell. 2023. Measuring the Success of Diffusion Models at Imitating Human Artists. arXiv preprint arXiv:2307.04028 (2023). · arXiv
  39. Stephen Casper, Kaivalya Hariharan, and Dylan Hadfield-Menell. 2022. Diagnostics for deep neural networks with automated copy/paste attacks. In NeurIPS ML Safety Workshop.
  40. Stephen Casper, Taylor Killian, Gabriel Kreiman, and Dylan Hadfield-Menell. 2023. Red Teaming with Mind Reading: White-Box Adversarial Policies Against RL Agents. (Oct. 2023). http://arxiv.org/abs/2209.02167 arXiv:2209.02167 [cs]. · arXiv
  41. Stephen Casper, Yuxiao Li, Jiawei Li, Tong Bu, Kevin Zhang, Kaivalya Hariharan, and Dylan Hadfield-Menell. 2023. Red Teaming Deep Neural Networks with Feature Synthesis Tools. (Sept. 2023). http://arxiv.org/abs/2302.10894 arXiv:2302.10894 [cs]. · arXiv
  42. Stephen Casper, Jason Lin, Joe Kwon, Gatlen Culp, and Dylan Hadfield-Menell. 2023. Explore, Establish, Exploit: Red Teaming Language Models from Scratch. arXiv preprint arXiv:2306.09442 (2023). · arXiv
  43. Stephen Casper, Max Nadeau, Dylan Hadfield-Menell, and Gabriel Kreiman. 2022. Robust feature-level adversaries are interpretability tools. Advances in Neural Information Processing Systems 35 (2022), 33093–33106.
  44. Stephen Casper Lennart Schulze Oam Patel and Dylan Hadfield-Menell. 2024. Defending Against Unforeseen Failure Modes with Latent Adversarial Training. arxiv:2403.05030 [cs.CR] · arXiv
  45. Patrick Chao, Alexander Robey, Edgar Dobriban, Hamed Hassani, George J Pappas, and Eric Wong. 2023. Jailbreaking Black Box Large Language Models in Twenty Queries. arXiv preprint arXiv:2310.08419 (2023). · arXiv
  46. PV Charan, Hrushikesh Chunduri, P Mohan Anand, and Sandeep K Shukla. 2023. From Text to MITRE Techniques: Exploring the Malicious Use of Large Language Models for Generating Cyber Attack Payloads. arXiv preprint arXiv:2305.15336 (2023). · arXiv
  47. 10.1145/3564284 · DOI · Paper page
  48. Xinyun Chen, Chang Liu, Bo Li, Kimberly Lu, and Dawn Song. 2017. Targeted backdoor attacks on deep learning systems using data poisoning. arXiv preprint arXiv:1712.05526 (2017). · arXiv
  49. Zhenpeng Chen, Jie M Zhang, Max Hort, Federica Sarro, and Mark Harman. 2022. Fairness testing: A comprehensive survey and analysis of trends. arXiv preprint arXiv:2207.10223 (2022). · arXiv
  50. 10.48550/arXiv.2103.04244 · DOI
  51. James Coe and Mustafa Atay. 2021. Evaluating impact of race in facial recognition across machine learning and deep learning algorithms. Computers 10, 9 (2021), 113. · DOI
  52. Alexis Conneau, German Kruszewski, Guillaume Lample, Loïc Barrault, and Marco Baroni. 2018. What you can cram into a single vector: Probing sentence embeddings for linguistic properties. arXiv preprint arXiv:1805.01070 (2018). · arXiv
  53. Hoagy Cunningham Aidan Ewart Logan Riggs Robert Huben and Lee Sharkey. 2023. Sparse Autoencoders Find Highly Interpretable Features in Language Models. (2023). arxiv:2309.08600 [cs.LG] · arXiv
  54. Arun Das and Paul Rad. 2020. Opportunities and challenges in explainable artificial intelligence (xai): A survey. arXiv preprint arXiv:2006.11371 (2020). · arXiv
  55. Tom Davidson, Jean-Stanislas Denain, Pablo Villalobos, and Guillem Bas. 2023. AI capabilities can be significantly improved without expensive retraining. (Dec. 2023). https://arxiv.org/abs/2312.07413v1 · arXiv
  56. Chunyuan Deng, Yilun Zhao, Xiangru Tang, Mark Gerstein, and Arman Cohan. 2023. Investigating Data Contamination in Modern Benchmarks for Large Language Models. arXiv preprint arXiv:2311.09783 (2023). · arXiv
  57. Gelei Deng, Yi Liu, Yuekang Li, Kailong Wang, Ying Zhang, Zefeng Li, Haoyu Wang, Tianwei Zhang, and Yang Liu. 2023. Jailbreaker: Automated Jailbreak Across Multiple Large Language Model Chatbots. arXiv preprint arXiv:2307.08715 (2023). · arXiv
  58. Mingkai Deng, Jianyu Wang, Cheng-Ping Hsieh, Yihan Wang, Han Guo, Tianmin Shu, Meng Song, Eric P Xing, and Zhiting Hu. 2022. Rlprompt: Optimizing discrete text prompts with reinforcement learning. arXiv preprint arXiv:2205.12548 (2022). · arXiv
  59. Roel Dobbe, Thomas Krendl Gilbert, and Yonatan Mintz. 2019. Hard choices in artificial intelligence: Addressing normative uncertainty through sociotechnical commitments. arXiv preprint arXiv:1911.09005 (2019). · arXiv
  60. Yinpeng Dong, Hang Su, Jun Zhu, and Fan Bao. 2017. Towards interpretable deep neural networks by leveraging adversarial examples. arXiv preprint arXiv:1708.05493 (2017). · arXiv
  61. Mengnan Du, Fengxiang He, Na Zou, Dacheng Tao, and Xia Hu. 2023. Shortcut learning of large language models in natural language understanding. Communications of the ACM (CACM) (2023).
  62. Javid Ebrahimi, Anyi Rao, Daniel Lowd, and Dejing Dou. 2017. Hotflip: White-box adversarial examples for text classification. arXiv preprint arXiv:1712.06751 (2017). · arXiv
  63. 10.1177/0093854818811379 · DOI · Paper page
  64. 10.1086/229939 · DOI · Paper page
  65. Lauren B. Edelman. 2016. Working Law: Courts, Corporations, and Symbolic Civil Rights. University of Chicago Press, Chicago, IL. https://press.uchicago.edu/ucp/books/book/chicago/W/bo24550454.html · DOI
  66. European Commission. 2021. Laying down harmonised rules on artificial intelligence (Artificial Intelligence Act) and amending certain union legislative acts. Eur Comm 106 (2021), 1–108.
  67. 10.1257/jep.10.3.103 · DOI
  68. Ross D Fuerman. 2009. Bernard Madoff and the solo auditor red flag. Journal of Forensic & Investigative Accounting 1, 1 (2009), 1–38.
  69. Yossi Gandelsman, Alexei A Efros, and Jacob Steinhardt. 2023. Interpreting CLIP’s Image Representation via Text-Based Decomposition. arXiv preprint arXiv:2310.05916 (2023). · arXiv
  70. Deep Ganguli, Liane Lovitt, Jackson Kernion, Amanda Askell, Yuntao Bai, Saurav Kadavath, Ben Mann, Ethan Perez, Nicholas Schiefer, Kamal Ndousse, 2022. Red teaming language models to reduce harms: Methods, scaling behaviors, and lessons learned. arXiv preprint arXiv:2209.07858 (2022). · arXiv
  71. 10.1038/s42256-020-00257-z · DOI · Paper page
  72. Mor Geva, Roei Schuster, Jonathan Berant, and Omer Levy. 2020. Transformer feed-forward layers are key-value memories. arXiv preprint arXiv:2012.14913 (2020). · arXiv
  73. Amirata Ghorbani and James Zou. 2020. Neuron Shapley: Discovering the Responsible Neurons. (2020). arxiv:2002.09815 [stat.ML] · arXiv
  74. Leilani H. Gilpin, David Bau, Ben Z. Yuan, Ayesha Bajwa, Michael Specter, and Lalana Kagal. 2019. Explaining Explanations: An Overview of Interpretability of Machine Learning. (Feb. 2019). http://arxiv.org/abs/1806.00069 arXiv:1806.00069 [cs, stat]. · arXiv
  75. 10.1016/j.intacc.2013.04.004 · DOI
  76. Shahriar Golchin and Mihai Surdeanu. 2023. Time travel in llms: Tracing data contamination in large language models. arXiv preprint arXiv:2308.08493 (2023). · arXiv
  77. Arieh Goldman and Benzion Barlev. 1974. The Auditor-Firm Conflict of Interests: Its Implications for Independence. The Accounting Review 49, 4 (1974), 707–718. https://www.jstor.org/stable/245049 Publisher: American Accounting Association.
  78. Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. 2014. Explaining and harnessing adversarial examples. arXiv preprint arXiv:1412.6572 (2014). · arXiv
  79. 10.1145/3531146.3533074 · DOI · Paper page
  80. Will Grathwohl, Dami Choi, Yuhuai Wu, Geoffrey Roeder, and David Duvenaud. 2017. Backpropagation through the void: Optimizing control variates for black-box gradient estimation. arXiv preprint arXiv:1711.00123 (2017). · arXiv
  81. Jarek Gryz and Marcin Rojszczak. 2021. Black box algorithms and the rights of individuals: No easy solution to the" explainability" problem. Internet Policy Review 10, 2 (2021), 1–24.
  82. Chuan Guo, Alexandre Sablayrolles, Hervé Jégou, and Douwe Kiela. 2021. Gradient-based adversarial attacks against text transformers. arXiv preprint arXiv:2104.13733 (2021). · arXiv
  83. Philipp Hacker, Johann Cordes, and Janina Rochon. 2023. Regulating Gatekeeper AI and Data: Transparency, Access, and Fairness under the DMA, the GDPR, and beyond. (Aug. 2023). http://arxiv.org/abs/2212.04997 arXiv:2212.04997 [cs]. · arXiv
  84. 10.1007/978-3-031-04083-2_17 · DOI
  85. 10.1109/MCI.2021.3129960 · DOI
  86. Julian Hazell. 2023. Large language models can be used to effectively scale spear phishing campaigns. arXiv preprint arXiv:2305.06972 (2023). · arXiv
  87. Pengcheng He, Xiaodong Liu, Jianfeng Gao, and Weizhu Chen. 2020. Deberta: Decoding-enhanced bert with disentangled attention. arXiv preprint arXiv:2006.03654 (2020). · arXiv
  88. Peter Henderson, Xuechen Li, Dan Jurafsky, Tatsunori Hashimoto, Mark A Lemley, and Percy Liang. 2023. Foundation models and fair use. arXiv preprint arXiv:2303.15715 (2023). · arXiv
  89. Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. 2020. Measuring massive multitask language understanding. arXiv preprint arXiv:2009.03300 (2020). · arXiv
  90. Dan Hendrycks Kevin Zhao Steven Basart Jacob Steinhardt and Dawn Song. 2021. Natural Adversarial Examples. (2021). arxiv:1907.07174 [cs.LG] · DOI · Paper page
  91. Evan Hernandez, Sarah Schwettmann, David Bau, Teona Bagashvili, Antonio Torralba, and Jacob Andreas. 2021. Natural language descriptions of deep visual features. In International Conference on Learning Representations.
  92. 10.1111/ablj.12134 · DOI
  93. Lei Huang, Weijiang Yu, Weitao Ma, Weihong Zhong, Zhangyin Feng, Haotian Wang, Qianglong Chen, Weihua Peng, Xiaocheng Feng, Bing Qin, and Ting Liu. 2023. A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions. (2023). arxiv:2311.05232 [cs.CL] · arXiv
  94. Evan Hubinger. 2020. An overview of 11 proposals for building safe advanced ai. arXiv preprint arXiv:2012.07532 (2020). · arXiv
  95. Evan Hubinger, Carson Denison, Jesse Mu, Mike Lambert, Meg Tong, Monte MacDiarmid, Tamera Lanham, Daniel M Ziegler, Tim Maxwell, Newton Cheng, 2024. Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training. arXiv preprint arXiv:2401.05566 (2024). · arXiv
  96. Andrew Ilyas, Logan Engstrom, Anish Athalye, and Jessy Lin. 2018. Black-box adversarial attacks with limited queries and information. In International conference on machine learning. PMLR, 2137–2146.
  97. Alon Jacovi, Avi Caciularu, Omer Goldman, and Yoav Goldberg. 2023. Stop uploading test data in plain text: Practical strategies for mitigating data contamination by evaluation benchmarks. arXiv preprint arXiv:2305.10160 (2023). · arXiv
  98. Samyak Jain, Robert Kirk, Ekdeep Singh Lubana, Robert P Dick, Hidenori Tanaka, Edward Grefenstette, Tim Rocktäschel, and David Scott Krueger. 2023. Mechanistically analyzing the effects of fine-tuning on procedurally defined tasks. arXiv preprint arXiv:2311.12786 (2023). · arXiv
  99. Haoming Jiang, Pengcheng He, Weizhu Chen, Xiaodong Liu, Jianfeng Gao, and Tuo Zhao. 2019. Smart: Robust and efficient fine-tuning for pre-trained natural language models through principled regularized optimization. arXiv preprint arXiv:1911.03437 (2019). · arXiv
  100. W Jeffrey Johnston and Stefano Fusi. 2023. Abstract representations emerge naturally in neural networks trained to perform multiple tasks. Nature Communications 14, 1 (2023), 1040. · DOI · Paper page
  101. Erik Jones, Anca Dragan, Aditi Raghunathan, and Jacob Steinhardt. 2023. Automatically Auditing Large Language Models via Discrete Optimization. arXiv preprint arXiv:2303.04381 (2023). · arXiv
  102. Joemon M Jose 2021. On fairness and interpretability. arXiv preprint arXiv:2106.13271 (2021). · arXiv
  103. Antonia Karamolegkou, Jiaang Li, Li Zhou, and Anders Søgaard. 2023. Copyright Violations and Large Language Models. arXiv preprint arXiv:2310.13771 (2023). · arXiv
  104. Rabimba Karanjai. 2022. Targeted phishing campaigns using large scale language models. arXiv preprint arXiv:2301.00665 (2022). · arXiv
  105. Max Kaufmann Daniel Kang Yi Sun Steven Basart Xuwang Yin Mantas Mazeika Akul Arora Adam Dziedzic Franziska Boenisch Tom Brown Jacob Steinhardt and Dan Hendrycks. 2023. Testing Robustness Against Unforeseen Adversaries. (2023). arxiv:1908.08016 [cs.LG] · arXiv
  106. 10.1109/ACCESS.2020.3006051 · DOI
  107. Mohd Ehmer Khan and Farmeena Khan. 2012. A comparative study of white box, black box and grey box testing techniques. International Journal of Advanced Computer Science and Applications 3, 6 (2012).
  108. Been Kim, Martin Wattenberg, Justin Gilmer, Carrie Cai, James Wexler, Fernanda Viegas, and Rory Sayres. 2018. Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV). In Proceedings of the 35th International Conference on Machine Learning. PMLR, 2668–2677. https://proceedings.mlr.press/v80/kim18d.html ISSN: 2640-3498.
  109. Megan Kinniment, Lucas Jun Koba Sato, Haoxing Du, Brian Goodrich, Max Hasin, Lawrence Chan, Luke Harold Miles, Tao R Lin, Hjalmar Wijk, Joel Burget, Aaron Ho, Elizabeth Barnes, and Paul Christiano. 2023. Evaluating Language-Model Agents on Realistic Autonomous Tasks. https://evals.alignment.org/language-model-pilot-report. (July 2023).
  110. 10.1007/s10489-022-04301-w · DOI
  111. Leonie Koessler and Jonas Schuett. 2023. Risk assessment at AGI companies: A review of popular risk assessment techniques from other safety-critical industries. (July 2023). https://arxiv.org/abs/2307.08823v1 · arXiv
  112. Noam Kolt. 2023. Algorithmic black swans. Washington University Law Review 101 (2023).
  113. 10.1109/MC.2021.3067225 · DOI · Paper page
  114. 10.2139/ssrn.448221 · DOI
  115. 10.48550/ARXIV.2402.06625 · DOI
  116. Satyapriya Krishna, Rahul Gupta, Apurv Verma, Jwala Dhamala, Yada Pruksachatkun, and Kai-Wei Chang. 2022. Measuring Fairness of Text Classifiers via Prediction Sensitivity. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 5830–5842.
  117. 10.1007/s13347-019-00372-9 · DOI
  118. Nupur Kumari, Mayank Singh, Abhishek Sinha, Harshitha Machiraju, Balaji Krishnamurthy, and Vineeth N Balasubramanian. 2019. Harnessing the vulnerability of latent layers in adversarially trained models. In Proceedings of the 28th International Joint Conference on Artificial Intelligence. 2779–2785.
  119. Nathan Lambert, Thomas Krendl Gilbert, and Tom Zick. 2023. Entangled Preferences: The History and Risks of Reinforcement Learning and Human Feedback. (2023). arxiv:2310.13595 [cs.CY] · arXiv
  120. Richard N Landers and Tara S Behrend. 2023. Auditing the AI auditors: A framework for evaluating fairness and bias in high stakes AI predictive models.American Psychologist 78, 1 (2023), 36.
  121. Raz Lapid, Ron Langberg, and Moshe Sipper. 2023. Open Sesame! Universal Black Box Jailbreaking of Large Language Models. arXiv preprint arXiv:2309.01446 (2023). · arXiv
  122. Andrew Lee, Xiaoyan Bai, Itamar Pres, Martin Wattenberg, Jonathan K Kummerfeld, and Rada Mihalcea. 2024. A Mechanistic Understanding of Alignment Algorithms: A Case Study on DPO and Toxicity. arXiv preprint arXiv:2401.01967 (2024). · arXiv
  123. Sharkey Lee Ghuidhir Clíodhna Ní Dan Braun Scheurer Jérémy Mikita Balesni Bushnaq Lucius Stix Charlotte and Marius Hobbhahn. 2023. A causal framework for AI Regulation and Auditing. (2023). · DOI
  124. 10.2139/ssrn.4531029 · DOI
  125. 10.1016/S0165-4101(00)00025-2 · DOI · Paper page
  126. Simon Lermen Charlie Rogers-Smith and Jeffrey Ladish. 2023. LoRA Fine-tuning Efficiently Undoes Safety Training in Llama 2-Chat 70B. (2023). arxiv:2310.20624 [cs.LG] · arXiv
  127. Haoran Li, Dadi Guo, Wei Fan, Mingshi Xu, and Yangqiu Song. 2023. Multi-step Jailbreaking Privacy Attacks on ChatGPT. arXiv preprint arXiv:2304.05197 (2023). · arXiv
  128. Jinfeng Li, Shouling Ji, Tianyu Du, Bo Li, and Ting Wang. 2018. Textbugger: Generating adversarial text against real-world applications. arXiv preprint arXiv:1812.05271 (2018). · arXiv
  129. 10.1609/aaai.v35i9.17022 · DOI
  130. Percy Liang, Rishi Bommasani, Tony Lee, Dimitris Tsipras, Dilara Soylu, Michihiro Yasunaga, Yian Zhang, Deepak Narayanan, Yuhuai Wu, Ananya Kumar, 2022. Holistic evaluation of language models. arXiv preprint arXiv:2211.09110 (2022). · arXiv
  131. Cheryl Linthicum, Austin L Reitenga, and Juan Manuel Sanchez. 2010. Social responsibility and corporate reputation: The case of the Arthur Andersen Enron audit failure. Journal of Accounting and Public Policy 29, 2 (2010), 160–176. · DOI
  132. Aiwei Liu, Honghai Yu, Xuming Hu, Shuang Li, Li Lin, Fukun Ma, Yawen Yang, and Lijie Wen. 2022. Character-level White-Box Adversarial Attacks against Transformers via Attachable Subwords Substitution. ArXiv abs/2210.17004 (2022). https://api.semanticscholar.org/CorpusID:253236900 · arXiv
  133. Xiaodong Liu, Hao Cheng, Pengcheng He, Weizhu Chen, Yu Wang, Hoifung Poon, and Jianfeng Gao. 2020. Adversarial training for large neural language models. arXiv preprint arXiv:2004.08994 (2020). · arXiv
  134. Xingbin Liu, Huafeng Kuang, Hong Liu, Xianming Lin, Yongjian Wu, and Rongrong Ji. 2023. Latent Feature Relation Consistency for Adversarial Robustness. arXiv preprint arXiv:2303.16697 (2023). · arXiv
  135. Yanpei Liu, Xinyun Chen, Chang Liu, and Dawn Song. 2016. Delving into transferable adversarial examples and black-box attacks. arXiv preprint arXiv:1611.02770 (2016). · arXiv
  136. Yi Liu, Gelei Deng, Zhengzi Xu, Yuekang Li, Yaowen Zheng, Ying Zhang, Lida Zhao, Tianwei Zhang, and Yang Liu. 2023. Jailbreaking chatgpt via prompt engineering: An empirical study. arXiv preprint arXiv:2305.13860 (2023). · arXiv
  137. Alexandra Sasha Luccioni and Joseph D Viviano. 2021. What’s in the Box? A Preliminary Analysis of Undesirable Content in the Common Crawl Corpus. arXiv preprint arXiv:2105.02732 (2021). · arXiv
  138. 10.1016/j.acra.2019.09.009 · DOI
  139. Samuel Marks Can Rager Eric J. Michaud Yonatan Belinkov David Bau and Aaron Mueller. 2024. Sparse Feature Circuits: Discovering and Editing Interpretable Causal Graphs in Language Models. arxiv:2403.19647 [cs.LG] · arXiv
  140. Samuel Marks and Max Tegmark. 2023. The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasets. (2023). arxiv:2310.06824 [cs.AI] · arXiv
  141. 10.1287/orsc.2015.1039 · DOI · Paper page
  142. Miljan Martic, Jan Leike, Andrew Trask, Matteo Hessel, Shane Legg, and Pushmeet Kohli. 2018. Scaling shared model governance via model splitting. arXiv preprint arXiv:1812.05979 (2018). · arXiv
  143. Ninareh Mehrabi, Fred Morstatter, Nripsuta Saxena, Kristina Lerman, and Aram Galstyan. 2021. A survey on bias and fairness in machine learning. ACM computing surveys (CSUR) 54, 6 (2021), 1–35.
  144. Jacob Metcalf, Emanuel Moss, Ranjit Singh, Emnet Tafese, and Elizabeth Anne Watkins. 2022. A relationship and not a thing: A relational approach to algorithmic accountability and assessment documentation. arXiv preprint arXiv:2203.01455 (2022). · arXiv
  145. Tim Miller. 2019. Explanation in artificial intelligence: Insights from the social sciences. Artificial intelligence 267 (2019), 1–38.
  146. Andrea Miotti and Akash Wasil. 2023. Taking control: Policies to address extinction risks from advanced AI. arXiv preprint arXiv:2310.20563 (2023). · arXiv
  147. 10.5465/amr.2006.19379621 · DOI
  148. Jesse Mu and Jacob Andreas. 2020. Compositional explanations of neurons. Advances in Neural Information Processing Systems 33 (2020), 17153–17163.
  149. 10.1007/s11023-021-09557-8 · DOI · Paper page
  150. 10.1007/s43681-023-00289-2 · DOI · Paper page
  151. 10.1007/s43681-023-00289-2 · DOI · Paper page
  152. 10.48550/arXiv.2311.10538 · DOI · Paper page
  153. Neel Nanda Lawrence Chan Tom Lieberum Jess Smith and Jacob Steinhardt. 2023. Progress measures for grokking via mechanistic interpretability. (2023). arxiv:2301.05217 [cs.LG] · arXiv
  154. 10.48550/arXiv.2311.17035 · DOI · Paper page
  155. Richard Ngo, Lawrence Chan, and Sören Mindermann. 2022. The alignment problem from a deep learning perspective. arXiv preprint arXiv:2209.00626 (2022). · arXiv
  156. Aaron L Nielson. 2018. Sticky Regulations. U. Chi. L. Rev. 85 (2018), 85.
  157. Genki Osada, Budrul Ahsan, Revoti Prasad Bora, and Takashi Nishide. 2022. Latent Space Virtual Adversarial Training for Supervised and Semi-Supervised Learning. IEICE TRANSACTIONS on Information and Systems 105, 3 (2022), 667–678. · DOI
  158. Lin Pan, Chung-Wei Hang, Avirup Sil, and Saloni Potdar. 2022. Improved text classification via contrastive adversarial training. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 36. 11130–11138.
  159. Nicolas Papernot, Fartash Faghri, Nicholas Carlini, Ian Goodfellow, Reuben Feinman, Alexey Kurakin, Cihang Xie, Yash Sharma, Tom Brown, Aurko Roy, 2016. Technical report on the cleverhans v2. 1.0 adversarial examples library. arXiv preprint arXiv:1610.00768 (2016). · arXiv
  160. Geon Yeong Park and Sang Wan Lee. 2021. Reliably fast adversarial training via latent adversarial perturbation. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 7758–7767.
  161. PCAOB. 2002. Sarbanes-Oxley Act of 2002. https://pcaobus.org/About/History/Documents/PDFs/Sarbanes_Oxley_Act_of_2002.pdf Public Law 107-204, 116 Stat. 745.
  162. Ethan Perez, Saffron Huang, Francis Song, Trevor Cai, Roman Ring, John Aslanides, Amelia Glaese, Nat McAleese, and Geoffrey Irving. 2022. Red teaming language models with language models. arXiv preprint arXiv:2202.03286 (2022). · arXiv
  163. Ethan Perez, Sam Ringer, Kamilė Lukošiūtė, Karina Nguyen, Edwin Chen, Scott Heiner, Craig Pettit, Catherine Olsson, Sandipan Kundu, Saurav Kadavath, 2022. Discovering Language Model Behaviors with Model-Written Evaluations. arXiv preprint arXiv:2212.09251 (2022). · arXiv
  164. Thomas Ploug and Søren Holm. 2021. Right to Contest AI Diagnostics: Defining Transparency and Explainability Requirements from a Patient’s Perspective. In Artificial Intelligence in Medicine. Springer, 1–12.
  165. Luiza Pozzobon, Beyza Ermis, Patrick Lewis, and Sara Hooker. 2023. On the Challenges of Using Black-Box APIs for Toxicity Evaluation in Research. arXiv preprint arXiv:2304.12397 (2023). · arXiv
  166. Archiki Prasad, Peter Hase, Xiang Zhou, and Mohit Bansal. 2022. Grips: Gradient-free, edit-based instruction search for prompting large language models. arXiv preprint arXiv:2203.07281 (2022). · arXiv
  167. Xiangyu Qi, Kaixuan Huang, Ashwinee Panda, Mengdi Wang, and Prateek Mittal. 2023. Visual Adversarial Examples Jailbreak Large Language Models. arXiv preprint arXiv:2306.13213 (2023). · arXiv
  168. Xiangyu Qi, Yi Zeng, Tinghao Xie, Pin-Yu Chen, Ruoxi Jia, Prateek Mittal, and Peter Henderson. 2023. Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!arXiv preprint arXiv:2310.03693 (2023). · arXiv
  169. Yaguan Qian, Qiqi Shao, Tengteng Yao, Bin Wang, Shouling Ji, Shaoning Zeng, Zhaoquan Gu, and Wassim Swaileh. 2021. Towards Speeding up Adversarial Training in Latent Spaces. arXiv preprint arXiv:2102.00662 (2021). · arXiv
  170. Yiting Qu, Xinyue Shen, Xinlei He, Michael Backes, Savvas Zannettou, and Yang Zhang. 2023. Unsafe diffusion: On the generation of unsafe images and hateful memes from text-to-image models. arXiv preprint arXiv:2305.13873 (2023). · arXiv
  171. Manish Raghavan and Pauline Kim. 2023. Limitations of the “Four-Fifths Rule” and Statistical Parity Tests for Measuring Fairness. https://openreview.net/forum?id=M2aNjwX4Ec&referrer=%5Bthe%20profile%20of%20Manish%20Raghavan%5D(%2Fprofile%3Fid%3D Manish_Raghavan1)
  172. 10.1145/3306618.3314244 · DOI · Paper page
  173. 10.1145/3571151 · DOI · Paper page
  174. 10.1145/3375627.3375820 · DOI · Paper page
  175. Javier Rando, Daniel Paleka, David Lindner, Lennart Heim, and Florian Tramèr. 2022. Red-teaming the stable diffusion safety filter. arXiv preprint arXiv:2210.04610 (2022). · arXiv
  176. Javier Rando and Florian Tramèr. 2023. Universal Jailbreak Backdoors from Poisoned Human Feedback. (2023). arxiv:2311.14455 [cs.AI] · arXiv
  177. Abhinav Rao Sachin Vashistha Atharva Naik Somak Aditya and Monojit Choudhury. 2023. Tricking LLMs into Disobedience: Understanding Analyzing and Preventing Jailbreaks. (2023). arxiv:2305.14965 [cs.CL] · arXiv
  178. Tilman Räuker, Anson Ho, Stephen Casper, and Dylan Hadfield-Menell. 2023. Toward transparent ai: A survey on interpreting the inner structures of deep neural networks. In 2023 IEEE Conference on Secure and Trustworthy Machine Learning (SaTML). IEEE, 464–483.
  179. Abhilasha Ravichander, Yonatan Belinkov, and Eduard Hovy. 2020. Probing the probing paradigm: Does probing accuracy entail task relevance?arXiv preprint arXiv:2005.00719 (2020). · arXiv
  180. 10.18653/v1/P19-1103 · DOI
  181. Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. 2016. Model-agnostic interpretability of machine learning. arXiv preprint arXiv:1606.05386 (2016). · arXiv
  182. 10.1145/3178876.3186143 · DOI
  183. Daniel Rodriguez Maffioli. 2023. Copyright in Generative AI training: Balancing Fair Use through Standardization and Transparency. Available at SSRN 4579322 (2023).
  184. Tom Roth, Yansong Gao, Alsharif Abuadbba, Surya Nepal, and Wei Liu. 2021. Token-Modification Adversarial Attacks for Natural Language Processing: A Survey. ArXiv abs/2103.00676 (2021). https://api.semanticscholar.org/CorpusID:232075640 · arXiv
  185. Cynthia Rudin. 2018. Please stop explaining black box models for high stakes decisions. Stat 1050 (2018), 26.
  186. Teerapong Sae-Lim and Suronapee Phoomvuthisarn. 2022. Weighted Token-Level Virtual Adversarial Training in Text Classification. In 2022 3rd International Conference on Pattern Recognition and Machine Learning (PRML). IEEE, 117–123.
  187. Jonas B Sandbrink. 2023. Artificial intelligence and biological misuse: Differentiating risks of language models and biological design tools. arXiv preprint arXiv:2306.13952 (2023). · arXiv
  188. Swami Sankaranarayanan, Arpit Jain, Rama Chellappa, and Ser Nam Lim. 2018. Regularizing deep networks using efficient layerwise adversarial training. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 32.
  189. Shibani Santurkar, Esin Durmus, Faisal Ladhak, Cinoo Lee, Percy Liang, and Tatsunori Hashimoto. 2023. Whose opinions do language models reflect?arXiv preprint arXiv:2303.17548 (2023). · arXiv
  190. 10.1109/CVPR42600.2020.00499 · DOI
  191. Rylan Schaeffer, Brando Miranda, and Sanmi Koyejo. 2023. Are Emergent Abilities of Large Language Models a Mirage? (2023). arxiv:2304.15004 [cs.AI] · arXiv
  192. Jérémy Scheurer, Mikita Balesni, and Marius Hobbhahn. 2023. Technical Report: Large Language Models can Strategically Deceive their Users when Put Under Pressure. arXiv preprint arXiv:2311.07590 (2023). · arXiv
  193. Jonas Schuett. 2022. Three lines of defense against risks from AI. arXiv preprint arXiv:2212.08364 (2022). · arXiv
  194. Jonas Schuett. 2023. AGI labs need an internal audit function. (May 2023). https://arxiv.org/abs/2305.17038v1 · arXiv
  195. Jonas Schuett, Noemi Dreksler, Markus Anderljung, David McCaffary, Lennart Heim, Emma Bluemke, and Ben Garfinkel. 2023. Towards best practices in AGI safety and governance: A survey of expert opinion. arXiv preprint arXiv:2305.07153 (2023). · arXiv
  196. Leo Schwinn David Dobre Stephan Günnemann and Gauthier Gidel. 2023. Adversarial Attacks and Defenses in Large Language Models: Old and New Threats. (2023). arxiv:2310.19737 [cs.AI] · arXiv
  197. Rusheb Shah Quentin Feuillade-Montixi Soroush Pour Arush Tagade Stephen Casper and Javier Rando. 2023. Scalable and Transferable Black-Box Jailbreaks for Language Models via Persona Modulation. (2023). arxiv:2311.03348 [cs.CL] · arXiv
  198. Nima Shahbazi, Yin Lin, Abolfazl Asudeh, and HV Jagadish. 2023. Representation Bias in Data: A Survey on Identification and Resolution Techniques. Comput. Surveys (2023).
  199. Lee Sharkey Clíodhna Ní Ghuidhir Dan Braun Jérémy Scheurer Mikita Balesni Lucius Bushnaq Charlotte Stix and Marius Hobbhahn. 2024. A Causal Framework for AI Regulation and Auditing. (2024). · DOI
  200. Mrinank Sharma Meg Tong Tomasz Korbak David Duvenaud Amanda Askell Samuel R. Bowman Newton Cheng Esin Durmus Zac Hatfield-Dodds Scott R. Johnston Shauna Kravec Timothy Maxwell Sam McCandlish Kamal Ndousse Oliver Rausch Nicholas Schiefer Da Yan Miranda Zhang and Ethan Perez. 2023. Towards Understanding Sycophancy in Language Models. (2023). arxiv:2310.13548 [cs.CL] · arXiv

200 of 261 shown

Cited by

Sorted by citations

Metadata

Authors
Casper, S., Ezell, C., Siegmann, C., Kolt, N., Curtis, T. L., Bucknall, B., Haupt, A., Wei, K., Scheurer, J., Hobbhahn, M., Sharkey, L., Krishna, S., Von Hagen, M., Alberti, S., Chan, A., Sun, Q., Gerovitch, M., Bau, D., Tegmark, M., Krueger, D., & Hadfield-Menell, D.
Affiliations
  • Massachusetts Institute of Technology, United States of America
  • Harvard University, USA
  • Massachusetts Institute of Technology, USA
  • University of Toronto, Canada
  • Centre for the Governance of AI, United Kingdom
  • Apollo Research, United Kingdom
  • Harvard University, United States of America
  • Stanford University, USA
  • Centre for the Governance of AI, Mila (Quebec AI Institute), Canada
  • Northeastern University, USA
  • University of Cambridge, United Kingdom
ORCID
Venue
The 2024 ACM Conference on Fairness, Accountability, and Transparency
Publisher
ACM
Published
2024-06-03
Pages
2254-2272
DOI
10.1145/3630106.3659037
Cited by
86
References
50
Licence
creativecommons.org/licenses/by/4.0/ (vor)

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review