Foundational Challenges in Assuring Alignment and Safety of Large Language Models
Open accessThe DOI matches its OpenAlex record; 206 of the 206 references it cites were found in a registry or an index.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
"Why Should I Trust You?": Explaining the Predictions of Any Classifier
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939778 · Cited by 17,555
Intriguing properties of neural networks
Szegedy, C., Zaremba, W., Sutskever, I., Bruna, J., Erhan, D., Goodfellow, I., & Fergus, R. (2013) · arXiv · doi:10.48550/arxiv.1312.6199 · Cited by 8,143
Explanation in Artificial Intelligence: Insights from the Social Sciences
Miller, T. (2017) · arXiv · doi:10.48550/arxiv.1706.07269 · Cited by 5,338
Survey of Hallucination in Natural Language Generation
Ji, Z., Lee, N., Frieske, R., Yu, T., Su, D., Xu, Y., Ishii, E., Bang, Y., Chen, D., Dai, W., et al. (2022) · arXiv · doi:10.48550/arxiv.2202.03629 · Cited by 4,981
Self-citationChain-of-Thought Prompting Elicits Reasoning in Large Language Models
Wei, J., Wang, X., Schuurmans, D., Bosma, M., Ichter, B., Xia, F., Chi, E., Le, Q., & Zhou, D. (2022) · arXiv · doi:10.48550/arxiv.2201.11903 · Cited by 4,263
Axiomatic attribution for deep networks
Sundararajan, M., Taly, A., & Yan, Q. (2017) · arXiv · doi:10.48550/arxiv.1703.01365 · Cited by 4,087
Llama 2: Open Foundation and Fine-Tuned Chat Models
Touvron, H., Martin, L., Stone, K., Albert, P., Almahairi, A., Babaei, Y., Bashlykov, N., Batra, S., Bhargava, P., Bhosale, S., et al. (2023) · arXiv · doi:10.48550/arxiv.2307.09288 · Cited by 2,604
Catastrophic forgetting in connectionist networks
French, R. (1999) · Trends in Cognitive Sciences · doi:10.1016/s1364-6613(99)01294-2 · Cited by 2,311
Generative Agents: Interactive Simulacra of Human Behavior
Park, J. S., O'Brien, J. C., Cai, C. J., Morris, M. R., Liang, P., & Bernstein, M. S. (2023) · arXiv · doi:10.48550/arxiv.2304.03442 · Cited by 2,173
“The Global Decline of the Labor Share”
Karabarbounis, L., & Neiman, B. (2014) · The Quarterly Journal of Economics · doi:10.1093/qje/qjt032 · Cited by 2,152
Large Language Models are Zero-Shot Reasoners
Kojima, T., Gu, S. S., Reid, M., Matsuo, Y., & Iwasawa, Y. (2022) · arXiv · doi:10.48550/arxiv.2205.11916 · Cited by 2,013
Learnability and the Vapnik-Chervonenkis dimension
Blumer, A., Ehrenfeucht, A., Haussler, D., & Warmuth, M. K. (1989) · Journal of the ACM · doi:10.1145/76359.76371 · Cited by 1,839
Sparks of Artificial General Intelligence: Early experiments with GPT-4
Bubeck, S., Chandrasekaran, V., Eldan, R., Gehrke, J., Horvitz, E., Kamar, E., Lee, P., Lee, Y. T., Li, Y., Lundberg, S., et al. (2023) · arXiv · doi:10.48550/arxiv.2303.12712 · Cited by 1,526
RANK ANALYSIS OF INCOMPLETE BLOCK DESIGNS: THE METHOD OF PAIRED COMPARISONS
BRADLEY, R. A., & TERRY, M. E. (1952) · Biometrika · doi:10.1093/biomet/39.3-4.324 · Cited by 1,434
Reputation helps solve the ‘tragedy of the commons’
Milinski, M., Semmann, D., & Krambeck, H.-J. (2002) · Nature · doi:10.1038/415424a · Cited by 1,371
Poisoning Attacks against Support Vector Machines
Biggio, B., Nelson, B., & Laskov, P. (2012) · arXiv · doi:10.48550/arxiv.1206.6389 · Cited by 1,331
Transferability in Machine Learning: from Phenomena to Black-Box Attacks using Adversarial Samples
Papernot, N., McDaniel, P., & Goodfellow, I. (2016) · arXiv · doi:10.48550/arxiv.1605.07277 · Cited by 1,326
Domain Generalization: A Survey
Zhou, K., Liu, Z., Qiao, Y., Xiang, T., & Loy, C. C. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3195549 · Cited by 1,315
Concrete Problems in AI Safety
Amodei, D., Olah, C., Steinhardt, J., Christiano, P., Schulman, J., & Mané, D. (2016) · arXiv · doi:10.48550/arxiv.1606.06565 · Cited by 1,314
Sanity Checks for Saliency Maps
Adebayo, J., Gilmer, J., Muelly, M., Goodfellow, I., Hardt, M., & Kim, B. (2018) · arXiv · doi:10.48550/arxiv.1810.03292 · Cited by 1,294
AutoPrompt: Eliciting Knowledge from Language Models with Automatically Generated Prompts
Shin, T., Razeghi, Y., Logan IV, R. L., Wallace, E., & Singh, S. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.346 · Cited by 1,279
Finetuned Language Models Are Zero-Shot Learners
Wei, J., Bosma, M., Zhao, V. Y., Guu, K., Yu, A. W., Lester, B., Du, N., Dai, A. M., & Le, Q. V. (2021) · arXiv · doi:10.48550/arxiv.2109.01652 · Cited by 1,101
Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoning
Chen, X., Liu, C., Li, B., Lu, K., & Song, D. (2017) · arXiv · doi:10.48550/arxiv.1712.05526 · Cited by 1,039
Training Compute-Optimal Large Language Models
Hoffmann, J., Borgeaud, S., Mensch, A., Buchatskaya, E., Cai, T., Rutherford, E., Casas, D. D. L., Hendricks, L. A., Welbl, J., Clark, A., et al. (2022) · arXiv · doi:10.48550/arxiv.2203.15556 · Cited by 929
Arjovsky, M., Bottou, L., Gulrajani, I., & Lopez-Paz, D. (2019) · arXiv · doi:10.48550/arxiv.1907.02893 · Cited by 927
Shen, X., Chen, Z., Backes, M., Shen, Y., & Zhang, Y. (2023) · arXiv · doi:10.48550/arxiv.2308.03825 · Cited by 861
The Malicious Use of Artificial Intelligence: Forecasting, Prevention, and Mitigation
Brundage, M., Avin, S., Clark, J., Toner, H., Eckersley, P., Garfinkel, B., Dafoe, A., Scharre, P., Zeitzoff, T., Filar, B., et al. (2018) · Apollo - University of Cambridge Repository · doi:10.17863/cam.22520 · Cited by 849
CTRL: A Conditional Transformer Language Model for Controllable Generation
Keskar, N. S., McCann, B., Varshney, L. R., Xiong, C., & Socher, R. (2019) · arXiv · doi:10.48550/arxiv.1909.05858 · Cited by 751
Artificial Intelligence, Algorithmic Pricing, and Collusion
Calvano, E., Calzolari, G., Denicolò, V., & Pastorello, S. (2020) · American Economic Review · doi:10.1257/aer.20190623 · Cited by 708
Direct Preference Optimization: Your Language Model is Secretly a Reward Model
Rafailov, R., Sharma, A., Mitchell, E., Manning, C. D., Ermon, S., & Finn, C. (2023) · Advances in Neural Information Processing Systems 36 · doi:10.52202/075280-2338 · Cited by 686
Explainable machine learning in deployment
Bhatt, U., Xiang, A., Sharma, S., Weller, A., Taly, A., Jia, Y., Ghosh, J., Puri, R., Moura, J. M. F., & Eckersley, P. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3375624 · Cited by 633
Before we knew it: an empirical study of zero-day attacks in the real world
Bilge, L., & Dumitraş, T. (2012) · Proceedings of the 2012 ACM conference on Computer and communications security · doi:10.1145/2382196.2382284 · Cited by 627
Coordinated Punishment of Defectors Sustains Cooperation and Can Proliferate When Rare
Boyd, R., Gintis, H., & Bowles, S. (2010) · Science · doi:10.1126/science.1183665 · Cited by 613
Understanding intermediate layers using linear classifier probes
Alain, G., & Bengio, Y. (2016) · arXiv · doi:10.48550/arxiv.1610.01644 · Cited by 581
Self-citationStatistical mechanics of learning from examples
Seung, H. S., Sompolinsky, H., & Tishby, N. (1992) · Physical Review A · doi:10.1103/physreva.45.6056 · Cited by 532
Jailbreaking Black Box Large Language Models in Twenty Queries
Chao, P., Robey, A., Dobriban, E., Hassani, H., Pappas, G. J., & Wong, E. (2023) · arXiv · doi:10.48550/arxiv.2310.08419 · Cited by 522
Aligning AI With Shared Human Values
Hendrycks, D., Burns, C., Basart, S., Critch, A., Li, J., Song, D., & Steinhardt, J. (2020) · arXiv · doi:10.48550/arxiv.2008.02275 · Cited by 517
Towards Faithfully Interpretable NLP Systems: How Should We Define and Evaluate Faithfulness?
Jacovi, A., & Goldberg, Y. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.386 · Cited by 503
Ethical and social risks of harm from Language Models
Weidinger, L., Mellor, J., Rauh, M., Griffin, C., Uesato, J., Huang, P.-S., Cheng, M., Glaese, M., Balle, B., Kasirzadeh, A., et al. (2021) · arXiv · doi:10.48550/arxiv.2112.04359 · Cited by 477
Self-citationShen, X., Chen, Z., Backes, M., Shen, Y., & Zhang, Y. (2024) · Proceedings of the 2024 on ACM SIGSAC Conference on Computer and Communications Security · doi:10.1145/3658644.3670388 · Cited by 441
Towards Reasoning in Large Language Models: A Survey
Huang, J., & Chang, K. C.-C. (2022) · arXiv · doi:10.48550/arxiv.2212.10403 · Cited by 428
Understanding the role of individual units in a deep neural network
Bau, D., Zhu, J.-Y., Strobelt, H., Lapedriza, A., Zhou, B., & Torralba, A. (2020) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1907375117 · Cited by 375
(2022) · doi:10.52202/068431-0278 · Cited by 358
Managing extreme AI risks amid rapid progress
Bengio, Y., Hinton, G., Yao, A., Song, D., Abbeel, P., Darrell, T., Harari, Y. N., Zhang, Y.-Q., Xue, L., Shalev-Shwartz, S., et al. (2024) · Science · doi:10.1126/science.adn0117 · Cited by 349
Self-citationLearning How to Ask: Querying LMs with Mixtures of Soft Prompts
Qin, G., & Eisner, J. (2021) · Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/2021.naacl-main.410 · Cited by 342
Can Large Language Models Be an Alternative to Human Evaluations?
Chiang, C.-H., & Lee, H.-Y. (2023) · arXiv · doi:10.48550/arxiv.2305.01937 · Cited by 331
Sociotechnical Harms of Algorithmic Systems: Scoping a Taxonomy for Harm Reduction
Shelby, R., Rismani, S., Henne, K., Moon, A., Rostamzadeh, N., Nicholas, P., Yilla-Akbari, N., Gallegos, J., Smart, A., Garcia, E., et al. (2023) · Proceedings of the 2023 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3600211.3604673 · Cited by 279
LIMA: Less Is More for Alignment
Zhou, C., Liu, P., Xu, P., Iyer, S., Sun, J., Mao, Y., Ma, X., Efrat, A., Yu, P., Yu, L., et al. (2023) · Advances in Neural Information Processing Systems 36 · doi:10.52202/075280-2400 · Cited by 275
LIMA: Less Is More for Alignment
Zhou, C., Liu, P., Xu, P., Iyer, S., Sun, J., Mao, Y., Ma, X., Efrat, A., Yu, P., Yu, L., et al. (2023) · arXiv · doi:10.48550/arxiv.2305.11206 · Cited by 275
Levels of AGI for Operationalizing Progress on the Path to AGI
Morris, M. R., Sohl-Dickstein, J., Fiedel, N., Warkentin, T., Dafoe, A., Faust, A., Farabet, C., & Legg, S. (2023) · arXiv · doi:10.48550/arxiv.2311.02462 · Cited by 274
Ferrara, E. (2023) · doi:10.2139/ssrn.4614223 · Cited by 266
Assuring the Machine Learning Lifecycle: Desiderata, Methods, and Challenges
Ashmore, R., Calinescu, R., & Paterson, C. (2022) · ACM Computing Surveys · doi:10.1145/3453444 · Cited by 256
A Survey on In-context Learning
Dong, Q., Li, L., Dai, D., Zheng, C., Ma, J., Li, R., Xia, H., Xu, J., Wu, Z., Liu, T., et al. (2023) · arXiv · doi:10.48550/arxiv.2301.00234 · Cited by 248
Leike, J., Martic, M., Krakovna, V., Ortega, P. A., Everitt, T., Lefrancq, A., Orseau, L., & Legg, S. (2017) · arXiv · doi:10.48550/arxiv.1711.09883 · Cited by 240
AI generates covertly racist decisions about people based on their dialect
Hofmann, V., Kalluri, P. R., Jurafsky, D., & King, S. (2024) · Nature · doi:10.1038/s41586-024-07856-5 · Cited by 233
Cooperation, Punishment, and the Evolution of Human Institutions
Henrich, J. (2006) · Science · doi:10.1126/science.1126398 · Cited by 233
Cooperative AI: machines must learn to find common ground
Dafoe, A., Bachrach, Y., Hadfield, G., Horvitz, E., Larson, K., & Graepel, T. (2021) · Nature · doi:10.1038/d41586-021-01170-0 · Cited by 232
Large Language Models Can Self-Improve
Huang, J., Gu, S., Hou, L., Wu, Y., Wang, X., Yu, H., & Han, J. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2023.emnlp-main.67 · Cited by 230
(2023) · doi:10.52202/075280-2425 · Cited by 200
Sánchez-Monedero, J., Dencik, L., & Edwards, L. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372849 · Cited by 196
Self-citationInference-Time Intervention: Eliciting Truthful Answers from a Language Model
Li, K., Patel, O., Viégas, F., Pfister, H., & Wattenberg, M. (2023) · arXiv · doi:10.48550/arxiv.2306.03341 · Cited by 184
Explaining neural scaling laws
Bahri, Y., Dyer, E., Kaplan, J., Lee, J., & Sharma, U. (2024) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2311878121 · Cited by 181
"Kelly is a Warm Person, Joseph is a Role Model": Gender Biases in LLM-Generated Reference Letters
Wan, Y., Pu, G., Sun, J., Garimella, A., Chang, K.-W., & Peng, N. (2023) · arXiv · doi:10.48550/arxiv.2310.09219 · Cited by 175
Using Large Language Models to Simulate Multiple Humans and Replicate Human Subject Studies
Aher, G., Arriaga, R. I., & Kalai, A. T. (2022) · arXiv · doi:10.48550/arxiv.2208.10264 · Cited by 161
Augmented Language Models: a Survey
Mialon, G., Dessì, R., Lomeli, M., Nalmpantis, C., Pasunuru, R., Raileanu, R., Rozière, B., Schick, T., Dwivedi-Yu, J., Celikyilmaz, A., et al. (2023) · arXiv · doi:10.48550/arxiv.2302.07842 · Cited by 160
Humans inherit artificial intelligence biases
Vicente, L., & Matute, H. (2023) · Scientific Reports · doi:10.1038/s41598-023-42384-8 · Cited by 158
Goldstein, J. A., Sastry, G., Musser, M., DiResta, R., Gentzel, M., & Sedova, K. (2023) · arXiv · doi:10.48550/arxiv.2301.04246 · Cited by 152
Scaling Laws for Autoregressive Generative Modeling
Henighan, T., Kaplan, J., Katz, M., Chen, M., Hesse, C., Jackson, J., Jun, H., Brown, T. B., Dhariwal, P., Gray, S., et al. (2020) · arXiv · doi:10.48550/arxiv.2010.14701 · Cited by 150
Visual Adversarial Examples Jailbreak Aligned Large Language Models
Qi, X., Huang, K., Panda, A., Henderson, P., Wang, M., & Mittal, P. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i19.30150 · Cited by 150
Combating misinformation in the age of LLMs: Opportunities and challenges
Chen, C., & Shu, K. (2024) · AI Magazine · doi:10.1002/aaai.12188 · Cited by 148
A General Language Assistant as a Laboratory for Alignment
Askell, A., Bai, Y., Chen, A., Drain, D., Ganguli, D., Henighan, T., Jones, A., Joseph, N., Mann, B., DasSarma, N., et al. (2021) · arXiv · doi:10.48550/arxiv.2112.00861 · Cited by 146
Deep Reinforcement Learning at the Edge of the Statistical Precipice
Agarwal, R., Schwarzer, M., Castro, P. S., Courville, A., & Bellemare, M. G. (2021) · arXiv · doi:10.48550/arxiv.2108.13264 · Cited by 142
The Hydra Effect: Emergent Self-repair in Language Model Computations
McGrath, T., Rahtz, M., Kramar, J., Mikulik, V., & Legg, S. (2023) · arXiv · doi:10.48550/arxiv.2307.15771 · Cited by 140
A Theory for Emergence of Complex Skills in Language Models
Arora, S., & Goyal, A. (2023) · arXiv · doi:10.48550/arxiv.2307.15936 · Cited by 129
AI and the Everything in the Whole Wide World Benchmark
Raji, I. D., Bender, E. M., Paullada, A., Denton, E., & Hanna, A. (2021) · arXiv · doi:10.48550/arxiv.2111.15366 · Cited by 127
Teaching Small Language Models to Reason
Magister, L. C., Mallinson, J., Adamek, J., Malmi, E., & Severyn, A. (2023) · Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) · doi:10.18653/v1/2023.acl-short.151 · Cited by 127
Transformer Feed-Forward Layers Build Predictions by Promoting Concepts in the Vocabulary Space
Geva, M., Caciularu, A., Wang, K. R., & Goldberg, Y. (2022) · arXiv · doi:10.48550/arxiv.2203.14680 · Cited by 123
Toward Transparent AI: A Survey on Interpreting the Inner Structures of Deep Neural Networks
Räuker, T., Ho, A., Casper, S., & Hadfield-Menell, D. (2023) · 2023 IEEE Conference on Secure and Trustworthy Machine Learning (SaTML) · doi:10.1109/satml54575.2023.00039 · Cited by 122
Self-citationScalable agent alignment via reward modeling: a research direction
Leike, J., Krueger, D., Everitt, T., Martic, M., Maini, V., & Legg, S. (2018) · arXiv · doi:10.48550/arxiv.1811.07871 · Cited by 121
Self-citationHui, X., Reshef, O., & Zhou, L. (2023) · doi:10.2139/ssrn.4527336 · Cited by 116
An Interpretability Illusion for BERT
Bolukbasi, T., Pearce, A., Yuan, A., Coenen, A., Reif, E., Viégas, F., & Wattenberg, M. (2021) · arXiv · doi:10.48550/arxiv.2104.07143 · Cited by 113
Challenges in Detoxifying Language Models
Welbl, J., Glaese, A., Uesato, J., Dathathri, S., Mellor, J., Hendricks, L. A., Anderson, K., Kohli, P., Coppin, B., & Huang, P.-S. (2021) · Findings of the Association for Computational Linguistics: EMNLP 2021 · doi:10.18653/v1/2021.findings-emnlp.210 · Cited by 112
What Can Transformers Learn In-Context? A Case Study of Simple Function Classes
Garg, S., Tsipras, D., Liang, P., & Valiant, G. (2022) · arXiv · doi:10.48550/arxiv.2208.01066 · Cited by 110
Larger language models do in-context learning differently
Wei, J., Wei, J., Tay, Y., Tran, D., Webson, A., Lu, Y., Chen, X., Liu, H., Huang, D., Zhou, D., et al. (2023) · arXiv · doi:10.48550/arxiv.2303.03846 · Cited by 109
DecodingTrust: A comprehensive assessment of trustworthiness in GPT models
Wang, B., Chen, W., Pei, H., Xie, C., Kang, M., Zhang, C., Xu, C., Xiong, Z., Dutta, R., Schaeffer, R., et al. (2023) · Advances in Neural Information Processing Systems 36 · doi:10.52202/075280-1361 · Cited by 107
Secret Collusion among AI Agents: Multi-Agent Deception via Steganography
Motwani, S., Baranchuk, M., Strohmeier, M., Bolina, V., Torr, P., Hammond, L., & Witt, C. D. (2024) · Advances in Neural Information Processing Systems 37 · doi:10.52202/079017-2336 · Cited by 106
Self-citationInadequacies of Large Language Model Benchmarks in the Era of Generative Artificial Intelligence
McIntosh, T. R., Susnjak, T., Arachchilage, N., Liu, T., Watters, P., & Halgamuge, M. N. (2024) · arXiv · doi:10.48550/arxiv.2402.09880 · Cited by 104
Linearity of Relation Decoding in Transformer Language Models
Hernandez, E., Sharma, A. S., Haklay, T., Meng, K., Wattenberg, M., Andreas, J., Belinkov, Y., & Bau, D. (2023) · arXiv · doi:10.48550/arxiv.2308.09124 · Cited by 98
Scalable Extraction of Training Data from (Production) Language Models
Nasr, M., Carlini, N., Hayase, J., Jagielski, M., Cooper, A. F., Ippolito, D., Choquette-Choo, C. A., Wallace, E., Tramèr, F., & Lee, K. (2023) · arXiv · doi:10.48550/arxiv.2311.17035 · Cited by 96
Self-citationUnsolved Problems in ML Safety
Hendrycks, D., Carlini, N., Schulman, J., & Steinhardt, J. (2021) · arXiv · doi:10.48550/arxiv.2109.13916 · Cited by 95
Mitchell, E., Lin, C., Bosselut, A., Finn, C., & Manning, C. D. (2021) · arXiv · doi:10.48550/arxiv.2110.11309 · Cited by 95
Shortcut Learning of Large Language Models in Natural Language Understanding
Du, M., He, F., Zou, N., Tao, D., & Hu, X. (2024) · Communications of the ACM · doi:10.1145/3596490 · Cited by 94
ARB: Advanced Reasoning Benchmark for Large Language Models
Sawada, T., Paleka, D., Havrilla, A., Tadepalli, P., Vidas, P., Kranias, A., Nay, J. J., Gupta, K., & Komatsuzaki, A. (2023) · arXiv · doi:10.48550/arxiv.2307.13692 · Cited by 92
Self-citationLanguage Models as Agent Models
Andreas, J. (2022) · arXiv · doi:10.48550/arxiv.2212.01681 · Cited by 92
Can Large Language Models Really Improve by Self-critiquing Their Own Plans?
Valmeekam, K., Marquez, M., & Kambhampati, S. (2023) · arXiv · doi:10.48550/arxiv.2310.08118 · Cited by 90
The WMDP Benchmark: Measuring and Reducing Malicious Use With Unlearning
Li, N., Pan, A., Gopal, A., Yue, S., Berrios, D., Gatti, A., Li, J. D., Dombrowski, A.-K., Goel, S., Phan, L., et al. (2024) · arXiv · doi:10.48550/arxiv.2403.03218 · Cited by 90
Self-citationAbstract representations emerge naturally in neural networks trained to perform multiple tasks
Johnston, W. J., & Fusi, S. (2023) · Nature Communications · doi:10.1038/s41467-023-36583-0 · Cited by 89
Debugging Tests for Model Explanations
Adebayo, J., Muelly, M., Liccardi, I., & Kim, B. (2020) · arXiv · doi:10.48550/arxiv.2011.05429 · Cited by 88
Transformers learn in-context by gradient descent
von Oswald, J., Niklasson, E., Randazzo, E., Sacramento, J., Mordvintsev, A., Zhmoginov, A., & Vladymyrov, M. (2022) · arXiv · doi:10.48550/arxiv.2212.07677 · Cited by 87
What learning algorithm is in-context learning? Investigations with linear models
Akyürek, E., Schuurmans, D., Andreas, J., Ma, T., & Zhou, D. (2022) · arXiv · doi:10.48550/arxiv.2211.15661 · Cited by 86
Chan, A., Ezell, C., Kaufmann, M., Wei, K., Hammond, L., Bradley, H., Bluemke, E., Rajkumar, N., Krueger, D., Kolt, N., et al. (2024) · The 2024 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3630106.3658948 · Cited by 84
Self-citationTheory of Mind for Multi-Agent Collaboration via Large Language Models
Li, H., Chong, Y. Q., Stepputtis, S., Campbell, J., Hughes, D., Lewis, M., & Sycara, K. (2023) · arXiv · doi:10.48550/arxiv.2310.10701 · Cited by 82
Knowledge Editing for Large Language Models: A Survey
Wang, S., Zhu, Y., Liu, H., Zheng, Z., Chen, C., & Li, J. (2025) · ACM Computing Surveys · doi:10.1145/3698590 · Cited by 79
GrIPS: Gradient-free, Edit-based Instruction Search for Prompting Large Language Models
Prasad, A., Hase, P., Zhou, X., & Bansal, M. (2023) · Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2023.eacl-main.277 · Cited by 78
Self-citationAlpacaFarm: A Simulation Framework for Methods that Learn from Human Feedback
Dubois, Y., Li, X., Taori, R., Zhang, T., Gulrajani, I., Ba, J., Guestrin, C., Liang, P., & Hashimoto, T. B. (2023) · arXiv · doi:10.48550/arxiv.2305.14387 · Cited by 78
How Language Model Hallucinations Can Snowball
Zhang, M., Press, O., Merrill, W., Liu, A., & Smith, N. A. (2023) · arXiv · doi:10.48550/arxiv.2305.13534 · Cited by 77
Mass-Editing Memory in a Transformer
Meng, K., Sharma, A. S., Andonian, A., Belinkov, Y., & Bau, D. (2022) · arXiv · doi:10.48550/arxiv.2210.07229 · Cited by 76
Recursively Summarizing Books with Human Feedback
Wu, J., Ouyang, L., Ziegler, D. M., Stiennon, N., Lowe, R., Leike, J., & Christiano, P. (2021) · arXiv · doi:10.48550/arxiv.2109.10862 · Cited by 76
Do Large Language Models Know What They Don’t Know?
Yin, Z., Sun, Q., Guo, Q., Wu, J., Qiu, X., & Huang, X. (2023) · Findings of the Association for Computational Linguistics: ACL 2023 · doi:10.18653/v1/2023.findings-acl.551 · Cited by 75
Do Large Language Models Know What They Don't Know?
Yin, Z., Sun, Q., Guo, Q., Wu, J., Qiu, X., & Huang, X. (2023) · arXiv · doi:10.48550/arxiv.2305.18153 · Cited by 75
Choi, M., Pei, J., Kumar, S., Shu, C., & Jurgens, D. (2023) · arXiv · doi:10.48550/arxiv.2305.14938 · Cited by 73
Categorizing Variants of Goodhart's Law
Manheim, D., & Garrabrant, S. (2018) · arXiv · doi:10.48550/arxiv.1803.04585 · Cited by 73
Valmeekam, K., Marquez, M., Olmo, A., Sreedharan, S., & Kambhampati, S. (2022) · arXiv · doi:10.48550/arxiv.2206.10498 · Cited by 70
Data Distributional Properties Drive Emergent In-Context Learning in Transformers
Chan, S., Santoro, A., Lampinen, A., Wang, J., Singh, A., Richemond, P., McClelland, J., & Hill, F. (2022) · Advances in Neural Information Processing Systems 35 · doi:10.52202/068431-1371 · Cited by 69
Data Distributional Properties Drive Emergent In-Context Learning in Transformers
Chan, S. C. Y., Santoro, A., Lampinen, A. K., Wang, J. X., Singh, A., Richemond, P. H., McClelland, J., & Hill, F. (2022) · arXiv · doi:10.48550/arxiv.2205.05055 · Cited by 69
Deduplicating Training Data Mitigates Privacy Risks in Language Models
Kandpal, N., Wallace, E., & Raffel, C. (2022) · arXiv · doi:10.48550/arxiv.2202.06539 · Cited by 68
Representation Engineering: A Top-Down Approach to AI Transparency
Zou, A., Phan, L., Chen, S., Campbell, J., Guo, P., Ren, R., Pan, A., Yin, X., Mazeika, M., Dombrowski, A.-K., et al. (2023) · arXiv · doi:10.48550/arxiv.2310.01405 · Cited by 65
Self-citationSparse Autoencoders Find Highly Interpretable Features in Language Models
Cunningham, H., Ewart, A., Riggs, L., Huben, R., & Sharkey, L. (2023) · arXiv · doi:10.48550/arxiv.2309.08600 · Cited by 61
Wu, Z., Qiu, L., Ross, A., Akyürek, E., Chen, B., Wang, B., Kim, N., Andreas, J., & Kim, Y. (2024) · Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers) · doi:10.18653/v1/2024.naacl-long.102 · Cited by 58
The Quantization Model of Neural Scaling
Michaud, E. J., Liu, Z., Girit, U., & Tegmark, M. (2023) · arXiv · doi:10.48550/arxiv.2303.13506 · Cited by 57
(2023) · doi:10.52202/075280-2687 · Cited by 57
Self-citationRisks from Learned Optimization in Advanced Machine Learning Systems
Hubinger, E., van Merwijk, C., Mikulik, V., Skalse, J., & Garrabrant, S. (2019) · arXiv · doi:10.48550/arxiv.1906.01820 · Cited by 53
Large Language Model Unlearning
Yao, Y., Xu, X., & Liu, Y. (2023) · arXiv · doi:10.48550/arxiv.2310.10683 · Cited by 53
Sclar, M., Choi, Y., Tsvetkov, Y., & Suhr, A. (2023) · arXiv · doi:10.48550/arxiv.2310.11324 · Cited by 52
Self-citationInterpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 small
Wang, K., Variengien, A., Conmy, A., Shlegeris, B., & Steinhardt, J. (2022) · arXiv · doi:10.48550/arxiv.2211.00593 · Cited by 52
Elhage, N., Hume, T., Olsson, C., Schiefer, N., Henighan, T., Kravec, S., Hatfield-Dodds, Z., Lasenby, R., Drain, D., Chen, C., et al. (2022) · arXiv · doi:10.48550/arxiv.2209.10652 · Cited by 51
Self-critiquing models for assisting human evaluators
Saunders, W., Yeh, C., Wu, J., Bills, S., Ouyang, L., Ward, J., & Leike, J. (2022) · arXiv · doi:10.48550/arxiv.2206.05802 · Cited by 50
The Ghost in the Machine has an American accent: value conflict in GPT-3
Johnson, R. L., Pistilli, G., Menédez-González, N., Duran, L. D. D., Panai, E., Kalpokiene, J., & Bertulfo, D. J. (2022) · arXiv · doi:10.48550/arxiv.2203.07785 · Cited by 50
Still no lie detector for language models: probing empirical and conceptual roadblocks
Levinstein, B. A., & Herrmann, D. A. (2025) · Philosophical Studies · doi:10.1007/s11098-023-02094-3 · Cited by 50
Greshake, K., Abdelnabi, S., Mishra, S., Endres, C., Holz, T., & Fritz, M. (2023) · arXiv · doi:10.48550/arxiv.2302.12173 · Cited by 50
Sociotechnical Safety Evaluation of Generative AI Systems
Weidinger, L., Rauh, M., Marchal, N., Manzini, A., Hendricks, L. A., Mateos-Garcia, J., Bergman, S., Kay, J., Griffin, C., Bariach, B., et al. (2023) · arXiv · doi:10.48550/arxiv.2310.11986 · Cited by 49
Towards Measuring the Representation of Subjective Global Opinions in Language Models
Durmus, E., Nguyen, K., Liao, T. I., Schiefer, N., Askell, A., Bakhtin, A., Chen, C., Hatfield-Dodds, Z., Hernandez, D., Joseph, N., et al. (2023) · arXiv · doi:10.48550/arxiv.2306.16388 · Cited by 48
Fundamental Limitations of Alignment in Large Language Models
Wolf, Y., Wies, N., Avnery, O., Levine, Y., & Shashua, A. (2023) · arXiv · doi:10.48550/arxiv.2304.11082 · Cited by 47
Kenton, Z., Everitt, T., Weidinger, L., Gabriel, I., Mikulik, V., & Irving, G. (2021) · arXiv · doi:10.48550/arxiv.2103.14659 · Cited by 45
Anticipating Safety Issues in E2E Conversational AI: Framework and Tooling
Dinan, E., Abercrombie, G., Bergman, A. S., Spruit, S., Hovy, D., Boureau, Y.-L., & Rieser, V. (2021) · arXiv · doi:10.48550/arxiv.2107.03451 · Cited by 44
Clever Hans or Neural Theory of Mind? Stress Testing Social Reasoning in Large Language Models
Shapira, N., Levy, M., Alavi, S. H., Zhou, X., Choi, Y., Goldberg, Y., Sap, M., & Shwartz, V. (2024) · Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2024.eacl-long.138 · Cited by 44
Self-citationCultural Incongruencies in Artificial Intelligence
Prabhakaran, V., Qadri, R., & Hutchinson, B. (2022) · arXiv · doi:10.48550/arxiv.2211.13069 · Cited by 42
Poisoning Language Models During Instruction Tuning
Wan, A., Wallace, E., Shen, S., & Klein, D. (2023) · arXiv · doi:10.48550/arxiv.2305.00944 · Cited by 42
Unveiling the Implicit Toxicity in Large Language Models
Wen, J., Ke, P., Sun, H., Zhang, Z., Li, C., Bai, J., & Huang, M. (2023) · arXiv · doi:10.48550/arxiv.2311.17391 · Cited by 40
Self-citationFine-Grained Human Feedback Gives Better Rewards for Language Model Training
Wu, Z., Hu, Y., Shi, W., Dziri, N., Suhr, A., Ammanabrolu, P., Smith, N. A., Ostendorf, M., & Hajishirzi, H. (2023) · arXiv · doi:10.48550/arxiv.2306.01693 · Cited by 40
Truthful AI: Developing and governing AI that does not lie
Evans, O., Cotton-Barratt, O., Finnveden, L., Bales, A., Balwit, A., Wills, P., Righetti, L., & Saunders, W. (2021) · arXiv · doi:10.48550/arxiv.2110.06674 · Cited by 39
Supervising strong learners by amplifying weak experts
Christiano, P., Shlegeris, B., & Amodei, D. (2018) · arXiv · doi:10.48550/arxiv.1810.08575 · Cited by 39
Choi, M., Pei, J., Kumar, S., Shu, C., & Jurgens, D. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2023.emnlp-main.699 · Cited by 38
PentestGPT: An LLM-empowered Automatic Penetration Testing Tool
Deng, G., Liu, Y., Mayoral-Vilches, V., Liu, P., Li, Y., Xu, Y., Zhang, T., Liu, Y., Pinzger, M., & Rass, S. (2023) · arXiv · doi:10.48550/arxiv.2308.06782 · Cited by 37
The Learnability of In-Context Learning
Wies, N., Levine, Y., & Shashua, A. (2023) · Advances in Neural Information Processing Systems 36 · doi:10.52202/075280-1593 · Cited by 36
Conservative Agency via Attainable Utility Preservation
Turner, A. M., Hadfield-Menell, D., & Tadepalli, P. (2020) · Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3375627.3375851 · Cited by 36
Getting aligned on representational alignment
Sucholutsky, I., Muttenthaler, L., Weller, A., Peng, A., Bobu, A., Kim, B., Love, B. C., Cueva, C. J., Grant, E., Groen, I., et al. (2023) · arXiv · doi:10.48550/arxiv.2310.13018 · Cited by 36
FOLIO: Natural Language Reasoning with First-Order Logic
Han, S., Schoelkopf, H., Zhao, Y., Qi, Z., Riddell, M., Zhou, W., Coady, J., Peng, D., Qiao, Y., Benson, L., et al. (2022) · arXiv · doi:10.48550/arxiv.2209.00840 · Cited by 35
Measuring Progress on Scalable Oversight for Large Language Models
Bowman, S. R., Hyun, J., Perez, E., Chen, E., Pettit, C., Heiner, S., Lukošiūtė, K., Askell, A., Jones, A., Chen, A., et al. (2022) · arXiv · doi:10.48550/arxiv.2211.03540 · Cited by 34
Dialect prejudice predicts AI decisions about people's character, employability, and criminality
Hofmann, V., Kalluri, P. R., Jurafsky, D., & King, S. (2024) · arXiv · doi:10.48550/arxiv.2403.00742 · Cited by 33
Benchmarking Foundation Models with Language-Model-as-an-Examiner
Bai, Y., Ying, J., Cao, Y., Lv, X., He, Y., Wang, X., Yu, J., Zeng, K., Xiao, Y., Lyu, H., et al. (2023) · arXiv · doi:10.48550/arxiv.2306.04181 · Cited by 33
Exploring Large Language Models for Communication Games: An Empirical Study on Werewolf
Xu, Y., Wang, S., Li, P., Luo, F., Wang, X., Liu, W., & Liu, Y. (2023) · arXiv · doi:10.48550/arxiv.2309.04658 · Cited by 32
Advanced Artificial Agents Intervene in the Provision of Reward
Cohen, M. K., Hutter, M., & Osborne, M. A. (2022) · AI Magazine · doi:10.1002/aaai.12064 · Cited by 30
KTO: Model Alignment as Prospect Theoretic Optimization
Ethayarajh, K., Xu, W., Muennighoff, N., Jurafsky, D., & Kiela, D. (2024) · arXiv · doi:10.48550/arxiv.2402.01306 · Cited by 29
Spear Phishing With Large Language Models
Hazell, J. (2023) · arXiv · doi:10.48550/arxiv.2305.06972 · Cited by 28
FANToM: A Benchmark for Stress-testing Machine Theory of Mind in Interactions
Kim, H., Sclar, M., Zhou, X., Bras, R., Kim, G., Choi, Y., & Sap, M. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2023.emnlp-main.890 · Cited by 27
Self-citationStudying Large Language Model Generalization with Influence Functions
Grosse, R., Bae, J., Anil, C., Elhage, N., Tamkin, A., Tajdini, A., Steiner, B., Li, D., Durmus, E., Perez, E., et al. (2023) · arXiv · doi:10.48550/arxiv.2308.03296 · Cited by 26
The Effects of Reward Misspecification: Mapping and Mitigating Misaligned Models
Pan, A., Bhatia, K., & Steinhardt, J. (2022) · arXiv · doi:10.48550/arxiv.2201.03544 · Cited by 25
Self-citationAI Research Considerations for Human Existential Safety (ARCHES)
Critch, A., & Krueger, D. (2020) · arXiv · doi:10.48550/arxiv.2006.04948 · Cited by 25
Self-citationSolving math word problems with process- and outcome-based feedback
Uesato, J., Kushman, N., Kumar, R., Song, F., Siegel, N., Wang, L., Creswell, A., Irving, G., & Higgins, I. (2022) · arXiv · doi:10.48550/arxiv.2211.14275 · Cited by 25
(2022) · doi:10.52202/068431-2344 · Cited by 24
Regulatory Markets: The Future of AI Governance
Hadfield, G. K., & Clark, J. (2023) · arXiv · doi:10.48550/arxiv.2304.04914 · Cited by 24
International Governance of Civilian AI: A Jurisdictional Certification Approach
Trager, R., Harack, B., Reuel, A., Carnegie, A., Heim, L., Ho, L., Kreps, S., Lall, R., Larter, O., hÉigeartaigh, S. Ó., et al. (2023) · arXiv · doi:10.48550/arxiv.2308.15514 · Cited by 23
Self-citationKirk, H. R., Vidgen, B., Röttger, P., & Hale, S. A. (2023) · arXiv · doi:10.48550/arxiv.2303.05453 · Cited by 22
LM-Polygraph: Uncertainty estimation for language models
Fadeeva, E., Vashurin, R., Tsvigun, A., Vazhentsev, A., Petrakov, S., Fedyanin, K., Vasilev, D., Goncharova, E., Panchenko, A., Panov, M., et al. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing: System Demonstrations · doi:10.18653/v1/2023.emnlp-demo.41 · Cited by 20
Identifying the Risks of LM Agents with an LM-Emulated Sandbox
Ruan, Y., Dong, H., Wang, A., Pitis, S., Zhou, Y., Ba, J., Dubois, Y., Maddison, C. J., & Hashimoto, T. (2023) · arXiv · doi:10.48550/arxiv.2309.15817 · Cited by 19
Conitzer, V., & Oesterheld, C. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i13.26791 · Cited by 19
Kenton, Z., Kumar, R., Farquhar, S., Richens, J., MacDermott, M., & Everitt, T. (2023) · Artificial Intelligence · doi:10.1016/j.artint.2023.103963 · Cited by 17
Mireshghallah, N., Kim, H., Zhou, X., Tsvetkov, Y., Sap, M., Shokri, R., & Choi, Y. (2023) · arXiv · doi:10.48550/arxiv.2310.17884 · Cited by 14
Self-citationRLPrompt: Optimizing Discrete Text Prompts with Reinforcement Learning
Deng, M., Wang, J., Hsieh, C.-P., Wang, Y., Guo, H., Shu, T., Song, M., Xing, E. P., & Hu, Z. (2022) · arXiv · doi:10.48550/arxiv.2205.12548 · Cited by 14
GPT-4 Is Too Smart To Be Safe: Stealthy Chat with LLMs via Cipher
Yuan, Y., Jiao, W., Wang, W., Huang, J.-T., He, P., Shi, S., & Tu, Z. (2023) · arXiv · doi:10.48550/arxiv.2308.06463 · Cited by 14
Watermarks in the Sand: Impossibility of Strong Watermarking for Generative Models
Zhang, H., Edelman, B. L., Francati, D., Venturi, D., Ateniese, G., & Barak, B. (2023) · arXiv · doi:10.48550/arxiv.2311.04378 · Cited by 12
Self-citationScalable and Transferable Black-Box Jailbreaks for Language Models via Persona Modulation
Shah, R., Feuillade--Montixi, Q., Pour, S., Tagade, A., Casper, S., & Rando, J. (2023) · arXiv · doi:10.48550/arxiv.2311.03348 · Cited by 12
Self-citationStyle Over Substance: Evaluation Biases for Large Language Models
Wu, M., & Aji, A. F. (2023) · arXiv · doi:10.48550/arxiv.2307.03025 · Cited by 11
The Unlocking Spell on Base LLMs: Rethinking Alignment via In-Context Learning
Lin, B. Y., Ravichander, A., Lu, X., Dziri, N., Sclar, M., Chandu, K., Bhagavatula, C., & Choi, Y. (2023) · arXiv · doi:10.48550/arxiv.2312.01552 · Cited by 11
Self-citationGPT-4 Doesn't Know It's Wrong: An Analysis of Iterative Prompting for Reasoning Problems
Stechly, K., Marquez, M., & Kambhampati, S. (2023) · arXiv · doi:10.48550/arxiv.2310.12397 · Cited by 11
Gopal, A., Helm-Burger, N., Justen, L., Soice, E. H., Tzeng, T., Jeyapragasan, G., Grimm, S., Mueller, B., & Esvelt, K. M. (2023) · arXiv · doi:10.48550/arxiv.2310.18233 · Cited by 10
The Authenticity Gap in Human Evaluation
Ethayarajh, K., & Jurafsky, D. (2022) · Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2022.emnlp-main.406 · Cited by 10
FireAct: Toward Language Agent Fine-tuning
Chen, B., Shu, C., Shareghi, E., Collier, N., Narasimhan, K., & Yao, S. (2023) · arXiv · doi:10.48550/arxiv.2310.05915 · Cited by 9
Elazar, Y., Bhagia, A., Magnusson, I., Ravichander, A., Schwenk, D., Suhr, A., Walsh, P., Groeneveld, D., Soldaini, L., Singh, S., et al. (2023) · arXiv · doi:10.48550/arxiv.2310.20707 · Cited by 9
Effective Prompt Extraction from Language Models
Zhang, Y., Carlini, N., & Ippolito, D. (2023) · arXiv · doi:10.48550/arxiv.2307.06865 · Cited by 8
Taken out of context: On measuring situational awareness in LLMs
Berglund, L., Stickland, A. C., Balesni, M., Kaufmann, M., Tong, M., Korbak, T., Kokotajlo, D., & Evans, O. (2023) · arXiv · doi:10.48550/arxiv.2309.00667 · Cited by 8
Self-citationHow to Catch an AI Liar: Lie Detection in Black-Box LLMs by Asking Unrelated Questions
Pacchiardi, L., Chan, A. J., Mindermann, S., Moscovitz, I., Pan, A. Y., Gal, Y., Evans, O., & Brauner, J. (2023) · arXiv · doi:10.48550/arxiv.2309.15840 · Cited by 8
Self-citationMassive Editing for Large Language Models via Meta Learning
Tan, C., Zhang, G., & Fu, J. (2023) · arXiv · doi:10.48550/arxiv.2311.04661 · Cited by 8
Tensor Trust: Interpretable Prompt Injection Attacks from an Online Game
Toyer, S., Watkins, O., Mendes, E. A., Svegliato, J., Bailey, L., Wang, T., Ong, I., Elmaaroufi, K., Abbeel, P., Darrell, T., et al. (2023) · arXiv · doi:10.48550/arxiv.2311.01011 · Cited by 7
Algorithmic progress in computer vision
Erdil, E., & Besiroglu, T. (2022) · arXiv · doi:10.48550/arxiv.2212.05153 · Cited by 7
Overthinking the Truth: Understanding how Language Models Process False Demonstrations
Halawi, D., Denain, J.-S., & Steinhardt, J. (2023) · arXiv · doi:10.48550/arxiv.2307.09476 · Cited by 7
Kenton, Z., Kumar, R., Farquhar, S., Richens, J., MacDermott, M., & Everitt, T. (2022) · arXiv · doi:10.48550/arxiv.2208.08345 · Cited by 7
Specific versus General Principles for Constitutional AI
Kundu, S., Bai, Y., Kadavath, S., Askell, A., Callahan, A., Chen, A., Goldie, A., Balwit, A., Mirhoseini, A., McLean, B., et al. (2023) · arXiv · doi:10.48550/arxiv.2310.13798 · Cited by 7
How FaR Are Large Language Models From Agents with Theory-of-Mind?
Zhou, P., Madaan, A., Potharaju, S. P., Gupta, A., McKee, K. R., Holtzman, A., Pujara, J., Ren, X., Mishra, S., Nematzadeh, A., et al. (2023) · arXiv · doi:10.48550/arxiv.2310.03051 · Cited by 6
A Logic for Expressing Log-Precision Transformers
Merrill, W., & Sabharwal, A. (2023) · Advances in Neural Information Processing Systems 36 · doi:10.52202/075280-2284 · Cited by 6
A Cost Analysis of Generative Language Models and Influence Operations
Musser, M. (2023) · arXiv · doi:10.48550/arxiv.2308.03740 · Cited by 6
Uncovering mesa-optimization algorithms in Transformers
von Oswald, J., Schlegel, M., Meulemans, A., Kobayashi, S., Niklasson, E., Zucchet, N., Scherrer, N., Miller, N., Sandler, M., Arcas, B. A. Y., et al. (2023) · arXiv · doi:10.48550/arxiv.2309.05858 · Cited by 5
Testing Language Model Agents Safely in the Wild
Naihin, S., Atkinson, D., Green, M., Hamadi, M., Swift, C., Schonholtz, D., Kalai, A. T., & Bau, D. (2023) · arXiv · doi:10.48550/arxiv.2311.10538 · Cited by 4
Governing Through the Cloud: The Intermediary Role of Compute Providers in AI Regulation
Heim, L., Fist, T., Egan, J., Huang, S., Zekany, S., Trager, R., Osborne, M. A., & Zilberman, N. (2024) · arXiv · doi:10.48550/arxiv.2403.08501 · Cited by 4
Zhang, Y., Zhang, F., Yang, Z., & Wang, Z. (2023) · arXiv · doi:10.48550/arxiv.2305.19420 · Cited by 4
Defending Against Unforeseen Failure Modes with Latent Adversarial Training
Casper, S., Schulze, L., Patel, O., & Hadfield-Menell, D. (2024) · arXiv · doi:10.48550/arxiv.2403.05030 · Cited by 3
Self-citationA Long Way to Go: Investigating Length Correlations in RLHF
Singhal, P., Goyal, T., Xu, J., & Durrett, G. (2023) · arXiv · doi:10.48550/arxiv.2310.03716 · Cited by 3
LLM360: Towards Fully Transparent Open-Source LLMs
Liu, Z., Qiao, A., Neiswanger, W., Wang, H., Tan, B., Tao, T., Li, J., Wang, Y., Sun, S., Pangarkar, O., et al. (2023) · arXiv · doi:10.48550/arxiv.2312.06550 · Cited by 3
Data Contamination Through the Lens of Time
Roberts, M., Thakur, H., Herlihy, C., White, C., & Dooley, S. (2023) · arXiv · doi:10.48550/arxiv.2310.10628 · Cited by 2
Cited by
Darwin Godel Machine: Open-Ended Evolution of Self-Improving Agents
Zhang, J., Hu, S., Lu, C., Lange, R., & Clune, J. (2025) · arXiv · doi:10.48550/arxiv.2505.22954 · Cited by 344
How secure is AI-generated code: a large-scale comparison of large language models
Tihanyi, N., Bisztray, T., Ferrag, M. A., Jain, R., & Cordeiro, L. C. (2025) · Empirical Software Engineering · doi:10.1007/s10664-024-10590-1 · Cited by 133
Generative language models exhibit social identity biases
Hu, T., Kyrychenko, Y., Rathje, S., Collier, N., van der Linden, S., & Roozenbeek, J. (2024) · Nature Computational Science · doi:10.1038/s43588-024-00741-1 · Cited by 100
Know Your Limits: A Survey of Abstention in Large Language Models
Wen, B., Yao, J., Feng, S., Xu, C., Tsvetkov, Y., Howe, B., & Wang, L. L. (2025) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00754 · Cited by 56
Liu, Y., Lo, S. K., Lu, Q., Zhu, L., Zhao, D., Xu, X., Harrer, S., & Whittle, J. (2025) · Journal of Systems and Software · doi:10.1016/j.jss.2024.112278 · Cited by 51
Metadata
- Authors
- Anwar, U., Saparov, A., Ramírez, J. R., Paleka, D., Turpin, M., Hase, P., Lubana, E. S., Jenner, E., Casper, S., Sourbut, O., Edelman, B. L., Zhang, Z., Günther, M., Korinek, A., Hernandez-Orallo, J., Hammond, L., Bigelow, E., Pan, A., Langosco, L., Korbak, T., Zhang, H. C., Zhong, R., ÓhÉigeartaigh, S. S., Recchia, G. L., Corsi, G., Chan, A., Anderljung, M., Edwards, L., Petrov, A., de Witt, C. A. S., Motwani, S., Bengio, Y., Chen, D., Torr, P. H. S., Albanie, S., Maharaj, T., Foerster, J. N., Tramèr, F., He, H., Kasirzadeh, A., Choi, Y., & Krueger, D.
- Venue
- arXiv (Cornell University)
- Publisher
- Cornell University
- Published
- 2024-01-01
- ISSN
- 2331-8422 (ISSN-L)
- OpenAlex
- W6947695150
- Cited by
- 17
- References
- 206
- Type
- posted content
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.