CoChat
Sign Up
PreprintCited by 36Ethics and Social Impacts of AIArtificial Intelligence ApplicationsExplainable Artificial Intelligence (XAI)

Can We Trust AI Benchmarks? An Interdisciplinary Review of Current Issues in AI Evaluation

Open access
Eriksson, Maria, Purificato, Erasmo, Noroozian, Arman, Vinagre, Joao, Chaslot, Guillaume, Gomez, Emilia, Fernandez-Llorca, David·2025·arXiv
DOI 10.48550/arxiv.2502.06559arXiv 2502.06559
Open this paper in CoChat — free accountOpen PDF

Abstract

Quantitative Artificial Intelligence (AI) Benchmarks have emerged as fundamental tools for evaluating the performance, capability, and safety of AI models and systems. Currently, they shape the direction of AI development and are playing an increasingly prominent role in regulatory frameworks. As their influence grows, however, so too does concerns about how and with what effects they evaluate highly sensitive topics such as capabilities, including high-impact capabilities, safety and systemic risks. This paper presents an interdisciplinary meta-review of about 100 studies that discuss shortcomings in quantitative benchmarking practices, published in the last 10 years. It brings together many fine-grained issues in the design and application of benchmarks (such as biases in dataset creation, inadequate documentation, data contamination, and failures to distinguish signal from noise) with broader sociotechnical issues (such as an over-focus on evaluating text-based AI models according to one-time testing logic that fails to account for how AI models are increasingly multimodal and interact with humans and other technical systems). Our review also highlights a series of systemic flaws in current benchmarking practices, such as misaligned incentives, construct validity issues, unknown unknowns, and problems with the gaming of benchmark results. Furthermore, it underscores how benchmark practices are fundamentally shaped by cultural, commercial and competitive dynamics that often prioritise state-of-the-art performance at the expense of broader societal concerns. By providing an overview of risks associated with existing benchmarking procedures, we problematise disproportionate trust placed in benchmarks and contribute to ongoing efforts to improve the accountability and relevance of quantitative AI benchmarks within the complexities of real-world scenarios.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2502.06559

Cites

0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

The publisher hasn’t deposited this paper’s reference list.

Research works printed but not linked: 78

  1. Norah Alzahrani, Hisham Abdullah Alyahya, Yazeed Alnumay, Sultan Alrashed, Shaykhah Alsubaie, Yusef Al- mushaykeh, Faisal Mirza, Nouf Alotaibi, Nora Altwairesh, Areeb Alowisheq, M. Saiful Bari, and Haidar Khan. When Benchmarks are Targets: Revealing the Sensitivity of Large Language Model Leaderboards, July 2024. URL http://arxiv.org/abs/2402.01781. · arXiv
  2. Varvara Arzt and Allan Hanbury. Beyond the Numbers: Transparency in Relation Extraction Benchmark Creation and Leaderboards, November 2024. URL http://arxiv.org/abs/2411.05224. · arXiv
  3. Michelle Bao, Angela Zhou, Samantha Zottola, Brian Brubach, Sarah Desmarais, Aaron Horowitz, Kristian Lum, and Suresh Venkatasubramanian. It's COMPASlicated: The Messy Relationship between RAI Datasets and Algorithmic Fairness Benchmarks, April 2022. URL http://arxiv.org/abs/2106.05498. · arXiv
  4. Thomas Bartz-Beielstein, Carola Doerr, Daan van den Berg, Jakob Bossek, Sowmya Chandrasekaran, Tome Eftimov, Andreas Fischbach, Pascal Kerschke, William La Cava, Manuel Lopez-Ibanez, Katherine M. Malan, Jason H. Moore, Boris Naujoks, Patryk Orzechowski, Vanessa Volz, Markus Wagner, and Thomas Weise. Benchmarking in Optimization: Best Practice and Open Issues, December 2020. URL http://arxiv.org/abs/2007.03488. · arXiv
  5. Stella Biderman, Hailey Schoelkopf, Lintang Sutawika, Leo Gao, Jonathan Tow, Baber Abbasi, Alham Fikri Aji, Pawan Sasanka Ammanamanchi, Sidney Black, Jordan Clive, Anthony DiPofi, Julen Etxaniz, Benjamin Fattori, Jessica Zosa Forde, Charles Foster, Jeffrey Hsu, Mimansa Jaiswal, Wilson Y. Lee, Haonan Li, Charles Lovering, Niklas Muennighoff, Ellie Pavlick, Jason Phang, Aviya Skowron, Samson Tan, Xiangru Tang, Kevin A. Wang, Genta Indra Winata, François Yvon, and Andy Zou. Lessons from the Trenches on Reproducible Evaluation of Language Models, May 2024. URL http://arxiv.org/abs/2405.14782. · arXiv
  6. Abeba Birhane, Ryan Steed, Victor Ojewale, Briana Vecchione, and Inioluwa Deborah Raji. AI auditing: The Broken Bus on the Road to AI Accountability, January 2024. URL http://arxiv.org/abs/2401.14462. · arXiv
  7. Isabelle Bruno. Benchmarking, pages 363-368. Springer Netherlands, Dordrecht, 2014. ISBN 978-94-007-0753-5. doi:10.1007/978-94-007-0753-5_170. URL https://doi.org/10.1007/978-94-007-0753-5_170. · DOI
  8. John Burden. Evaluating AI Evaluation: Perils and Prospects, July 2024. URL https://arxiv.org/abs/2407.
  9. Robert C. Camp. Benchmarking : The Search for Industry Best Practices That Lead to Superior Performance. Quality Press, the University of Michigan, 1989.
  10. Yupeng Chang, Xu Wang, Jindong Wang, Yuan Wu, Linyi Yang, Kaijie Zhu, Hao Chen, Xiaoyuan Yi, Cunxiang Wang, Yidong Wang, Wei Ye, Yue Zhang, Yi Chang, Philip S. Yu, Qiang Yang, and Xing Xie. A Survey on Evaluation of Large Language Models, December 2023. URL http://arxiv.org/abs/2307.03109. · arXiv
  11. Z. Cheng, CS. Pang, P. Wang, and et al. How to report and benchmark emerging field-effect transistors. Nature Electronics, 5:416-423, 2020. URL https://doi.org/10.1038/s41928-022-00798-8. · DOI
  12. François Chollet. On the Measure of Intelligence, November 2019. URL http://arxiv.org/abs/1911.01547. · arXiv
  13. Mostafa Dehghani, Yi Tay, Alexey A. Gritsenko, Zhe Zhao, Neil Houlsby, Fernando Diaz, Donald Metzler, and Oriol Vinyals. The Benchmark Lottery, July 2021. URL http://arxiv.org/abs/2107.07002. · arXiv
  14. Remi Denton, Alex Hanna, Razvan Amironesei, Andrew Smart, Hilary Nicole, and Morgan Klaus Scheuerman. Bringing the People Back In: Contesting Benchmark Machine Learning Datasets, July 2020. URL http://arxiv. org/abs/2007.07399.
  15. Kawin Ethayarajh and Dan Jurafsky. Utility is in the Eye of the User: A Critique of NLP Leaderboards, March 2021. URL http://arxiv.org/abs/2009.13888. · arXiv
  16. Simon Frieder, Jonas Bayer, Katherine M. Collins, Julius Berner, Jacob Loader, András Juhász, Fabian Ruehle, Sean Welleck, Gabriel Poesia, Ryan-Rhys Griffiths, Adrian Weller, Anirudh Goyal, Thomas Lukasiewicz, and Timothy Gowers. Data for Mathematical Copilots: Better Ways of Presenting Proofs for Machine Learning, December 2024. URL http://arxiv.org/abs/2412.15184. · arXiv
  17. Timnit Gebru, Jamie Morgenstern, Briana Vecchione, Jennifer Wortman Vaughan, Hanna Wallach, Hal Daumé III, and Kate Crawford. Datasheets for Datasets, December 2021. URL http://arxiv.org/abs/1803.09010. · arXiv
  18. Aryo Pradipta Gema, Joshua Ong Jun Leang, Giwon Hong, Alessio Devoto, Alberto Carlo Maria Mancino, Rohit Saxena, Xuanli He, Yu Zhao, Xiaotang Du, Mohammad Reza Ghasemi Madani, Claire Barale, Robert McHardy, Joshua Harris, Jean Kaddour, Emile van Krieken, and Pasquale Minervini. Are We Done with MMLU?, June 2024. URL http://arxiv.org/abs/2406.04127. · arXiv
  19. Emilia Gomez, Porcaro Lorenzo, Pedro Frau Amar, and Joao Vinagre. Diversity in artificial intelligence conferences. Publications Office of the European Union JRC137550, Publications Office, 2024. URL https://data.europa. eu/doi/10.2760/796551. · DOI
  20. Ryan Greenblatt, Carson Denison, Benjamin Wright, Fabien Roger, Monte MacDiarmid, Sam Marks, Johannes Treutlein, Tim Belonax, Jack Chen, David Duvenaud, Akbir Khan, Julian Michael, Sören Mindermann, Ethan Perez, Linda Petrini, Jonathan Uesato, Jared Kaplan, Buck Shlegeris, Samuel R. Bowman, and Evan Hubinger. Alignment faking in large language models, December 2024. URL http://arxiv.org/abs/2412.14093. · arXiv
  21. Philipp Guldimann, Alexander Spiridonov, Robin Staab, Nikola Jovanović, Mark Vero, Velko Vechev, Anna Gue- orguieva, Mislav Balunović, Nikola Konstantinov, Pavol Bielik, Petar Tsankov, and Martin Vechev. COMPL-AI Framework: A Technical Interpretation and LLM Benchmarking Suite for the EU Artificial Intelligence Act, October 2024. URL http://arxiv.org/abs/2410.07959. · arXiv
  22. Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Measuring Massive Multitask Language Understanding, January 2021. URL http://arxiv.org/abs/2009.03300. · arXiv
  23. J.L. Henning. Spec cpu2000: measuring cpu performance in the new millennium. Computer, 33(7):28-35, 2000. doi:10.1109/2.869367. · DOI
  24. Sara Hooker. On the Limitations of Compute Thresholds as a Governance Strategy, July 2024. URL http://arxiv. org/abs/2407.05694.
  25. Dietmar Jannach and Christine Bauer. Escaping the McNamara Fallacy: Toward More Impactful Recommender Systems Research. AI Magazine, 41(4):79-95, December 2020. ISSN 0738-4602, 2371-9621. doi:10.1609/aimag.v41i4.5312. URL https://onlinelibrary.wiley.com/doi/10.1609/aimag.v41i4.5312. · DOI
  26. Florian Jaton. The Constitution of Algorithms: Ground-Truthing, Programming, Formulating. Inside Technology. The MIT Press, Cambridge, 2021. ISBN 978-0-262-54214-2 978-0-262-36323-5.
  27. Sayash Kapoor, Benedikt Stroebl, Zachary S. Siegel, Nitya Nadgir, and Arvind Narayanan. AI Agents That Matter, July 2024. URL http://arxiv.org/abs/2407.01502. · arXiv
  28. Bernard Koch, Emily Denton, Alex Hanna, and Jacob G. Foster. Reduced, Reused and Recycled: The Life of a Dataset in Machine Learning Research, December 2021. URL http://arxiv.org/abs/2112.01716. · arXiv
  29. Bernard J. Koch and David Peterson. From Protoscience to Epistemic Monoculture: How Benchmarking Set the Stage for the Deep Learning Revolution, April 2024. URL http://arxiv.org/abs/2404.06647. · arXiv
  30. Travis LaCroix and Alexandra Sasha Luccioni. Metaethical Perspectives on 'Benchmarking' AI Ethics, April 2022. URL http://arxiv.org/abs/2204.05151. · arXiv
  31. Gavin Leech, Juan J. Vazquez, Niclas Kupper, Misha Yagudin, and Laurence Aitchison. Questionable practices in machine learning, July 2024. URL https://arxiv.org/abs/2407.12220v2. · arXiv
  32. Percy Liang, Rishi Bommasani, Tony Lee, Dimitris Tsipras, Dilara Soylu, Michihiro Yasunaga, Yian Zhang, Deepak Narayanan, Yuhuai Wu, Ananya Kumar, Benjamin Newman, Binhang Yuan, Bobby Yan, Ce Zhang, Christian Cosgrove, Christopher D. Manning, Christopher Ré, Diana Acosta-Navas, Drew A. Hudson, Eric Zelikman, Esin Durmus, Faisal Ladhak, Frieda Rong, Hongyu Ren, Huaxiu Yao, Jue Wang, Keshav Santhanam, Laurel Orr, Lucia Zheng, Mert Yuksekgonul, Mirac Suzgun, Nathan Kim, Neel Guha, Niladri Chatterji, Omar Khattab, Peter Henderson, Qian Huang, Ryan Chi, Sang Michael Xie, Shibani Santurkar, Surya Ganguli, Tatsunori Hashimoto, Thomas Icard, Tianyi Zhang, Vishrav Chaudhary, William Wang, Xuechen Li, Yifan Mai, Yuhui Zhang, and Yuta Koreeda. Holistic Evaluation of Language Models, October 2023. URL http://arxiv.org/abs/2211.09110. · arXiv
  33. Q. Vera Liao and Ziang Xiao. Rethinking Model Evaluation as Narrowing the Socio-Technical Gap, June 2023. URL http://arxiv.org/abs/2306.03100. · arXiv
  34. Thomas Liao, Rohan Taori, Inioluwa Deborah Raji, and Ludwig Schmidt. Are we learning yet? a meta review of evaluation failures across machine learning. In Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 2), 2021. URL https://openreview.net/forum?id=mPducS1MsEK.
  35. Pengfei Liu, Jinlan Fu, Yang Xiao, Weizhe Yuan, Shuaicheng Chang, Junqi Dai, Yixin Liu, Zihuiwen Ye, Zi-Yi Dou, and Graham Neubig. ExplainaBoard: An Explainable Leaderboard for NLP, July 2021. URL http://arxiv.org/ abs/2104.06387. · arXiv
  36. Kristian Lum, Jacy Reese Anthis, Chirag Nagpal, and Alexander D'Amour. Bias in Language Models: Beyond Trick Tests and Toward RUTEd Evaluation, February 2024. URL https://arxiv.org/abs/2402.12649v1. · arXiv
  37. Peter Mattson, Christine Cheng, Gregory Diamos, Cody Coleman, Paulius Micikevicius, David Patterson, Han- lin Tang, Gu-Yeon Wei, Peter Bailis, Victor Bittorf, David Brooks, Dehao Chen, Debo Dutta, Udit Gupta, Kim Hazelwood, Andy Hock, Xinyuan Huang, Daniel Kang, David Kanter, Naveen Kumar, Jeffery Liao, Deepak Narayanan, Tayo Oguntebi, Gennady Pekhimenko, Lillian Pentecost, Vijay Janapa Reddi, Taylor Ro- bie, Tom St John, Carole-Jean Wu, Lingjie Xu, Cliff Young, and Matei Zaharia. Mlperf training benchmark. In I. Dhillon, D. Papailiopoulos, and V. Sze, editors, Proceedings of Machine Learning and Systems, vol- ume 2, pages 336-349, 2020a. URL https://proceedings.mlsys.org/paper_files/paper/2020/file/ 411e39b117e885341f25efb8912945f7-Paper.pdf.
  38. Timothy R. McIntosh, Teo Susnjak, Nalin Arachchilage, Tong Liu, Paul Watters, and Malka N. Halgamuge. Inadequacies of Large Language Model Benchmarks in the Era of Generative Artificial Intelligence, October 2024. URL http://arxiv.org/abs/2402.09880. · arXiv
  39. Alexander Meinke, Bronson Schoen, Jérémy Scheurer, Mikita Balesni, Rusheb Shah, and Marius Hobbhahn. Frontier Models are Capable of In-context Scheming, December 2024. URL https://arxiv.org/abs/2412.04984v1. · arXiv
  40. Julian Michael, Ari Holtzman, Alicia Parrish, Aaron Mueller, Alex Wang, Angelica Chen, Divyam Madaan, Nikita Nangia, Richard Yuanzhe Pang, Jason Phang, and Samuel R. Bowman. What Do NLP Researchers Believe? Results of the NLP Community Metasurvey, August 2022. URL http://arxiv.org/abs/2208.12852. · arXiv
  41. Marc Miltenberger, Steven Arzt, Philipp Holzinger, and Julius Näumann. Benchmarking the Benchmarks. In Proceedings of the ACM Asia Conference on Computer and Communications Security, pages 387-400, Melbourne VIC Australia, July 2023. ACM. ISBN 9798400700989. doi:10.1145/3579856.3582830. URL https://dl.acm. org/doi/10.1145/3579856.3582830. · DOI
  42. Moran Mizrahi, Guy Kaplan, Dan Malkin, Rotem Dror, Dafna Shahaf, and Gabriel Stanovsky. State of What Art? A Call for Multi-Prompt LLM Evaluation, May 2024. URL http://arxiv.org/abs/2401.00595. · arXiv
  43. Dylan Mulvin. Proxies: The Cultural Work of Standing In. Infrastructures. The MIT Press, Cambridge, 2021. ISBN 978-0-262-04514-8 978-0-262-36624-3.
  44. Milad Nasr, Nicholas Carlini, Jonathan Hayase, Matthew Jagielski, A. Feder Cooper, Daphne Ippolito, Christopher A. Choquette-Choo, Eric Wallace, Florian Tramèr, and Katherine Lee. Scalable Extraction of Training Data from (Production) Language Models, November 2023b. URL http://arxiv.org/abs/2311.17035. · arXiv
  45. Arman Noroozian. Evaluating Hosting Provider Security Through Abuse Data and the Creation of Metrics. Dissertation (TU Delft), 2020. ISBN: 9789065624451.
  46. Luke Oakden-Rayner, Jared Dunnmon, Gustavo Carneiro, and Christopher Ré. Hidden Stratification Causes Clinically Meaningful Failures in Machine Learning for Medical Imaging, November 2019. URL http://arxiv.org/abs/ 1909.12475.
  47. Victor Ojewale, Ryan Steed, Briana Vecchione, Abeba Birhane, and Inioluwa Deborah Raji. Towards AI Accountability Infrastructure: Gaps and Opportunities in AI Audit Tooling, March 2024. URL http://arxiv.org/abs/2402. 17861.
  48. Will Orr and Kate Crawford. The social construction of datasets: On the practices, processes, and challenges of dataset creation for machine learning. New Media & Society, 26(9):4955-4972, 2024a.
  49. Will Orr and Kate Crawford. Building Better Datasets: Seven Recommendations for Responsible Design from Dataset Creators, August 2024b. URL http://arxiv.org/abs/2409.00252. · arXiv
  50. Oxford English Dictionary. Benchmark. meaning and use, 2017. URL https://www.oed.com/dictionary/ benchmark_n?tab=meaning_and_use&tl=true.
  51. Lorenzo Pacchiardi, Marko Tesic, Lucy G. Cheke, and José Hernández-Orallo. Leaving the barn door open for Clever Hans: Simple features predict LLM benchmark answers, October 2024. URL http://arxiv.org/abs/2410. 11672.
  52. Rolf Pfister and Hansueli Jud. Understanding and Benchmarking Artificial Intelligence: OpenAI's o3 Is Not AGI, January 2025. URL http://arxiv.org/abs/2501.07458. · arXiv
  53. Wessel Poelman and Miryam de Lhoneux. The Roles of English in Evaluating Multilingual Language Models, December 2024. URL http://arxiv.org/abs/2412.08392. · arXiv
  54. Xiangyu Qi, Yi Zeng, Tinghao Xie, Pin-Yu Chen, Ruoxi Jia, Prateek Mittal, and Peter Henderson. Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!, October 2023. URL http://arxiv.org/abs/2310.03693. · arXiv
  55. Inioluwa Deborah Raji, Emily Denton, Emily M. Bender, Alex Hanna, and Amandalynne Paullada. AI and the everything in the whole wide world benchmark. In Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 2), 2021. URL https://openreview.net/forum?id=j6NxpQbREA1.
  56. Maribeth Rauh, Nahema Marchal, Arianna Manzini, Lisa Anne Hendricks, Ramona Comanescu, Canfer Akbulut, Tom Stepleton, Juan Mateos-Garcia, Stevie Bergman, Jackie Kay, Conor Griffin, Ben Bariach, Iason Gabriel, Verena Rieser, William Isaac, and Laura Weidinger. Gaps in the Safety Evaluation of Generative AI. Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society, 7:1200-1217, October 2024. ISSN 3065-8365. doi:10.1609/aies.v7i1.31717. URL https://ojs.aaai.org/index.php/AIES/article/view/31717. · DOI
  57. Richard Ren, Steven Basart, Adam Khoja, Alice Gatti, Long Phan, Xuwang Yin, Mantas Mazeika, Alexander Pan, Gabriel Mukobi, Ryan Hwang Kim, Stephen Fitz, and Dan Hendrycks. Safetywashing: Do AI safety benchmarks actually measure safety progress? In The Thirty-eight Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2024. URL https://openreview.net/forum?id=YagfTP3RK6.
  58. Anka Reuel, Amelia Hardy, Chandler Smith, Max Lamparth, Malcolm Hardy, and Mykel Kochenderfer. Betterbench: Assessing AI benchmarks, uncovering issues, and establishing best practices. In The Thirty-eight Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2024. URL https://openreview.net/ forum?id=hcOq2buakM.
  59. Manley Roberts, Himanshu Thakur, Christine Herlihy, Colin White, and Samuel Dooley. Data Contamination Through the Lens of Time, October 2023. URL http://arxiv.org/abs/2310.10628. · arXiv
  60. Paul Röttger, Fabio Pernisi, Bertie Vidgen, and Dirk Hovy. SafetyPrompts: a Systematic Review of Open Datasets for Evaluating and Improving Large Language Model Safety, April 2024. URL http://arxiv.org/abs/2404.05399. · arXiv
  61. David Schlangen. Targeting the Benchmark: On Methodology in Current Natural Language Processing Research, July 2020. URL http://arxiv.org/abs/2007.04792. · arXiv
  62. D Sculley, Jasper Snoek, Ali Rahimi, and Alex Wiltschko. Winner's Curse? On Pace, Progress, and Empirical Rigor. Vancouver, BC, Canada, 2018. URL https://openreview.net/pdf?id=rJWF0Fywf.
  63. Aarohi Srivastava, Abhinav Rastogi, Abhishek Rao, Abu Awal Md Shoeb, Abubakar Abid, Adam Fisch, Adam R. Brown, Adam Santoro, Aditya Gupta, Adrià Garriga-Alonso, Agnieszka Kluska, Aitor Lewkowycz, Akshat Agarwal, Alethea Power, Alex Ray, Alex Warstadt, et al. Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models, June 2023. URL http://arxiv.org/abs/2206.04615. · arXiv
  64. Isabelle Stengers. Another science is possible: a manifesto for slow science. Polity press, Cambridge, 2018. ISBN 978-1-5095-2180-7.
  65. Marilyn Strathern. 'Improving ratings': audit in the British University system. European Review, 5(3):305-321, July 1997. ISSN 10627987, 1234981X. doi:10.1002/(SICI)1234-981X(199707)5:3<305::AID-EURO184>3.0.CO;2- 4. URL https://www.cambridge.org/core/product/identifier/S1062798700002660/type/journal_ article. · DOI · Paper page
  66. Arjun Subramonian, Xingdi Yuan, Hal Daumé III, and Su Lin Blodgett. It Takes Two to Tango: Navigating Conceptual- izations of NLP Tasks and Measurements of Performance, May 2023. URL http://arxiv.org/abs/2305.09022. · arXiv
  67. Nandan Thakur, Nils Reimers, Andreas Rücklé, Abhishek Srivastava, and Iryna Gurevych. BEIR: A heterogeneous benchmark for zero-shot evaluation of information retrieval models. In Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 2), 2021. URL https://openreview.net/forum? id=wCu6T5xFjeJ.
  68. Kushal Tirumala, Aram Markosyan, Luke Zettlemoyer, and Armen Aghajanyan. Memorization Without Overfitting: Analyzing the Training Dynamics of Large Language Models. In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh, editors, Advances in Neural Information Processing Systems, volume 35, pages 38274-38290.
  69. Curran Associates, Inc., 2022. URL https://proceedings.neurips.cc/paper_files/paper/2022/file/ fa0509f4dab6807e2cb465715bf2d249-Paper-Conference.pdf.
  70. Dimitris Tsipras, Shibani Santurkar, Logan Engstrom, Andrew Ilyas, and Aleksander Madry. From ImageNet to Image Classification: Contextualizing Progress on Benchmarks, May 2020. URL http://arxiv.org/abs/2005.11295. · arXiv
  71. Keyon Vafa, Justin Y. Chen, Ashesh Rambachan, Jon Kleinberg, and Sendhil Mullainathan. Evaluating the World Model Implicit in a Generative Model, November 2024. URL http://arxiv.org/abs/2406.03689. · arXiv
  72. Laura Weidinger, Maribeth Rauh, Nahema Marchal, Arianna Manzini, Lisa Anne Hendricks, Juan Mateos-Garcia, Stevie Bergman, Jackie Kay, Conor Griffin, Ben Bariach, Iason Gabriel, Verena Rieser, and William Isaac. Sociotechnical Safety Evaluation of Generative AI Systems, October 2023. URL http://arxiv.org/abs/2310.11986. · arXiv
  73. Teun van der Weij, Felix Hofstätter, Ollie Jaffe, Samuel F. Brown, and Francis Rhys Ward. AI Sandbagging: Language Models can Strategically Underperform on Evaluations, June 2024. URL http://arxiv.org/abs/2406.07358. · arXiv
  74. Cheng Xu, Shuhao Guan, Derek Greene, and M.-Tahar Kechadi. Benchmark Data Contamination of Large Language Models: A Survey, June 2024a. URL http://arxiv.org/abs/2406.04244. · arXiv
  75. Ruijie Xu, Zengzhi Wang, Run-Ze Fan, and Pengfei Liu. Benchmarking Benchmark Leakage in Large Language Models, April 2024b. URL http://arxiv.org/abs/2404.18824. · arXiv
  76. Shuo Yang, Wei-Lin Chiang, Lianmin Zheng, Joseph E. Gonzalez, and Ion Stoica. Rethinking Benchmark and Contamination for Language Models with Rephrased Samples, November 2023. URL http://arxiv.org/abs/ 2311.04850.
  77. Lifan Yuan, Yangyi Chen, Ganqu Cui, Hongcheng Gao, Fangyuan Zou, Xingyi Cheng, Heng Ji, Zhiyuan Liu, and Maosong Sun. Revisiting Out-of-distribution Robustness in NLP: Benchmark, Analysis, and LLMs Evaluations. 37th Conference on Neural Information Processing Systems (NeurIPS 2023) Track on Datasets and Benchmarks, 2023.
  78. Andy K. Zhang, Kevin Klyman, Yifan Mai, Yoav Levine, Yian Zhang, Rishi Bommasani, and Percy Liang. Language model developers should report train-test overlap, October 2024. URL http://arxiv.org/abs/2410.08385. · arXiv

Cited by

Sorted by citations

Related

Authors
Eriksson, M., Purificato, E., Noroozian, A., Vinagre, J., Chaslot, G., Gomez, E., & Fernandez-Llorca, D.
Affiliations
European Commission (BE)
ORCID
Purificato, Erasmo
Venue
arXiv
Publisher
arXiv
Published
2025
Volume
8
Issue
1
Pages
850-864
ISSN
3065-8365 (ISSN-L)
DOI
10.48550/arxiv.2502.06559
arXiv
2502.06559
OpenAlex
W4415227752
Cited by
35
References
0
Citations by year
2026: 25 · 2025: 10
FWCI
35.77
Type
conference paper
Language
en
Open access
Licence
arxiv.org/licenses/nonexclusive-distrib/1.0/
Locations
  • Proceedings of the AAAI/ACM Conference on AI Ethics and Society (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • AAAI Publications (The Association for the Advancement of Artificial Intelligence (AAAI)) (submittedVersion)
  • arXiv (Cornell University)
Topics
  • Social Sciences › Safety Research › Ethics and Social Impacts of AI
  • Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence Applications
  • Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
Keywords
AI benchmarks, AI evaluation, data bias, contaminated data, signal-to-noise ratio, multimodal AI, human-AI interaction, AI safety, systemic risk, construct validity, incentive misalignment, unknown unknowns, AI regulation, accountability

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review