CoChat
Sign Up
Cited by 110Topic ModelingNatural Language Processing TechniquesAdversarial Robustness in Machine Learning

The DOI matches its Crossref record; 1 of the 1 references it cites were found in a registry or an index.

Measuring what Matters: Construct Validity in Large Language Model Benchmarks

Open access
Bean, Andrew M., Kearns, Ryan Othniel, Romanou, Angelika, Hafner, Franziska Sofia, Mayne, Harry, Batzner, Jan, Foroutan Eghlidi, Negar, Schmitz, Chris, Korgul, Karolina, Batra, Hunar, Deb, Oishi, Beharry, Emma, Emde, Cornelius, Foster, Thomas, Gausen, Anna, Grandury, María, Han, Sophia, Hofmann, Valentin, Ibrahim, Lujain, Kim, Hazel, Kirk, Hannah Rose, Lin, Fangru, Liu, Gabrielle, Luettgau, Lennart, Magomere, Jabez, Rystrøm, Jonathan, Sotnikova, Anna, Yang, Yushi, Zhao, Yilun, Bibi, Adel, Bosselut, Antoine, Clark, Ronald, Cohan, Arman, Foerster, Jakob, Gal, Yarin, Hale, Scott, Raji, Deborah, Summerfield, Christopher, Torr, Philip, Ududec, Cozmin, Rocher, Luc, Mahdi, Adam·2025·Advances in Neural Information Processing Systems 38
DOI 10.52202/085713-0590
Open this paper in CoChat — free accountOpen PDF

Abstract

Evaluating large language models (LLMs) is crucial for both assessing their capabilities and identifying safety or robustness issues prior to deployment. Reliably measuring abstract and complex phenomena such as 'safety' and 'robustness' requires strong construct validity, that is, having measures that represent what matters to the phenomenon. With a team of 29 expert reviewers, we conduct a systematic review of 445 LLM benchmarks from leading conferences in natural language processing and machine learning. Across the reviewed articles, we find patterns related to the measured phenomena, tasks, and scoring metrics which undermine the validity of the resulting claims. To address these shortcomings, we provide eight key recommendations and detailed actionable guidance to researchers and practitioners in developing LLM benchmarks.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.52202/085713-0590

Cites

1 of 1 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Benchmark creators often fail to assess whether their benchmark's construct validity might be compromised by arbitrary format effects or prompt-related biases.

    “Moreover, benchmark creators often fail to assess whether their benchmark's construct validity might be compromised by arbitrary format effects or prompt-related biases (Bean et al., 2025).”

    From Brittlebench: Quantifying LLM robustness via prompt sensitivity (2026) · cites this paper as Bean et al., 2025

    The “Justify construct validity” section says only 53.4% of reviewed benchmarks justify their validity. Section 5.2 recommends assessing format constraints, and section 5.3 notes that minor prompt variations can affect LLM results. Our copy does not establish how often creators specifically fail to assess format effects or prompt-related biases. (LLM, conf=0.82)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Part of this paper’s reference list could not be split into single entries.

Flagged first, then by citations

Research works printed but not linked: 38

  1. B. Hutchinson, N. Rostamzadeh, C. Greer, K. Heller, and V. Prabhakaran. "Evaluation Gaps in Machine Learning Practice". In: Proceedings of the 2022 ACM Conference on Fairness, Ac- countability, and Transparency. FAccT '22. New York, NY, USA: Association for Computing Machinery, 2022.
  2. R. T. McCoy, S. Yao, D. Friedman, M. Hardy, and T. L. Griffiths. "Embers of Autoregression: Understanding Large Language Models Through the Problem They Are Trained to Solve". In: arXiv:2309.13638 (2023). · arXiv
  3. J. Hernández-Orallo. The Measure of All Minds: Evaluating Natural and Artificial Intelligence. Cambridge: Cambridge University Press, 2017. ISBN: 978-1-107-15301-1.
  4. S. R. Bowman and G. Dahl. "What Will It Take to Fix Benchmarking in Natural Language Understanding?" In: Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Ed. by K. Toutanova et al. Online: Association for Computational Linguistics, 2021.
  5. L. J. Cronbach and P. E. Meehl. "Construct Validity in Psychological Tests". In: Psychological Bulletin (1955).
  6. T. Gebru et al. "Datasheets for Datasets". In: Communications of the ACM (2021).
  7. B. Nevo. "Face Validity Revisited". In: Journal of Educational Measurement (1985).
  8. D. T. Campbell and D. W. Fiske. "Convergent and Discriminant Validation by the Multitrait- Multimethod Matrix". In: Psychological Bulletin (1959).
  9. A. Wang et al. "SuperGLUE: A Stickier Benchmark for General-Purpose Language Under- standing Systems". In: Advances in Neural Information Processing Systems. Ed. by H. Wallach, H. Larochelle, A. Beygelzimer, E. Fox, and R. Garnett. Vol. 32. Curran Associates, Inc., 2019.
  10. J. Wei et al. "Emergent Abilities of Large Language Models". In: Transactions on Machine Learning Research (2022).
  11. K. Cobbe et al. Training Verifiers to Solve Math Word Problems. arXiv:2110.14168 [cs]. 2021. · arXiv
  12. Y. Ren, H. Ye, H. Fang, X. Zhang, and G. Song. "ValueBench: Towards Comprehensively Evaluating Value Orientations and Understanding of Large Language Models". In: Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Ed. by L.-W. Ku, A. Martins, and V. Srikumar. Bangkok, Thailand: Association for Computational Linguistics, 2024.
  13. M. U. Nasir, S. James, and J. Togelius. "GameTraversalBenchmark: Evaluating Planning Abilities Of Large Language Models Through Traversing 2D Game Maps". In: Advances in Neural Information Processing Systems. Ed. by A. Globerson et al. Vol. 37. Curran Associates, Inc., 2024.
  14. X. Liu et al. "AgentBench: Evaluating LLMs as Agents". In: The Twelfth International Conference on Learning Representations. 2024.
  15. Z. Ouyang et al. "CliMedBench: A Large-Scale Chinese Benchmark for Evaluating Medical Large Language Models in Clinical Scenarios". In: Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing. Ed. by Y. Al-Onaizan, M. Bansal, and Y.-N. Chen. Miami, Florida, USA: Association for Computational Linguistics, 2024.
  16. N. Shah, Z. Genc, and D. Araci. "StackEval: Benchmarking LLMs in Coding Assistance". In: Advances in Neural Information Processing Systems. Ed. by A. Globerson et al. Vol. 37. Curran Associates, Inc., 2024.
  17. Y. Hou et al. "WikiContradict: A Benchmark for Evaluating LLMs on Real-World Knowledge Conflicts from Wikipedia". In: Advances in Neural Information Processing Systems. Ed. by A. Globerson et al. Vol. 37. Curran Associates, Inc., 2024.
  18. S. Chang et al. "Dr.Spider: A Diagnostic Evaluation Benchmark towards Text-to-SQL Robust- ness". In: The Eleventh International Conference on Learning Representations. 2023.
  19. H. Kim et al. "FANToM: A Benchmark for Stress-testing Machine Theory of Mind in Interac- tions". In: Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing. Ed. by H. Bouamor, J. Pino, and K. Bali. Singapore: Association for Computational Linguistics, 2023.
  20. S. Yao, H. Chen, J. Yang, and K. Narasimhan. "WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agents". In: Advances in Neural Information Processing Systems. Ed. by S. Koyejo et al. Vol. 35. Curran Associates, Inc., 2022.
  21. X. Wang et al. "SciBench: Evaluating College-Level Scientific Problem-Solving Abilities of Large Language Models". In: Proceedings of the 41st International Conference on Machine Learning. Ed. by R. Salakhutdinov et al. Vol. 235. Proceedings of Machine Learning Research. PMLR, 2024.
  22. K.-H. Huang et al. "Embrace Divergence for Richer Insights: A Multi-document Summariza- tion Benchmark and a Case Study on Summarizing Diverse Information from News Articles". In: Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers). Ed. by K. Duh, H. Gomez, and S. Bethard. Mexico City, Mexico: Association for Computational Linguistics, 2024.
  23. A. Ajith et al. "LitSearch: A Retrieval Benchmark for Scientific Literature Search". In: Pro- ceedings of the 2024 Conference on Empirical Methods in Natural Language Processing.
  24. Ed. by Y. Al-Onaizan, M. Bansal, and Y.-N. Chen. Miami, Florida, USA: Association for Computational Linguistics, 2024.
  25. S. Diao et al. "Doolittle: Benchmarks and Corpora for Academic Writing Formalization". In: Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing.
  26. Ed. by H. Bouamor, J. Pino, and K. Bali. Singapore: Association for Computational Linguistics, 2023.
  27. Y. Huang et al. "DA-Code: Agent Data Science Code Generation Benchmark for Large Language Models". In: Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing. Ed. by Y. Al-Onaizan, M. Bansal, and Y.-N. Chen. Miami, Florida, USA: Association for Computational Linguistics, 2024.
  28. J. Wang et al. "A User-Centric Multi-Intent Benchmark for Evaluating Large Language Mod- els". In: Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing. Ed. by Y. Al-Onaizan, M. Bansal, and Y.-N. Chen. Miami, Florida, USA: Associa- tion for Computational Linguistics, 2024.
  29. A. Bean et al. "LINGOLY: A Benchmark of Olympiad-Level Linguistic Reasoning Puzzles in Low Resource and Extinct Languages". In: Advances in Neural Information Processing Systems. Ed. by A. Globerson et al. Vol. 37. Curran Associates, Inc., 2024.
  30. M. Ho et al. "WikiWhy: Answering and Explaining Cause-and-Effect Questions". In: The Eleventh International Conference on Learning Representations. 2023.
  31. Y. Ruan, C. J. Maddison, and T. Hashimoto. "Observational Scaling Laws and the Predictabil- ity of Langauge Model Performance". In: The Thirty-eighth Annual Conference on Neural Information Processing Systems. 2024.
  32. F. Maia Polo et al. "tinyBenchmarks: evaluating LLMs with fewer examples". In: Proceedings of the 41st International Conference on Machine Learning. Ed. by R. Salakhutdinov et al. Vol. 235. Proceedings of Machine Learning Research. PMLR, 2024.
  33. M. Mizrahi et al. "State of What Art? A Call for Multi-Prompt LLM Evaluation". In: Transac- tions of the Association for Computational Linguistics (2024).
  34. P. Maini, H. Jia, N. Papernot, and A. Dziedzic. "LLM Dataset Inference: Did You Train on My Dataset?" In: Advances in Neural Information Processing Systems (2024).
  35. N. Sahoo et al. "IndiBias: A Benchmark Dataset to Measure Social Biases in Language Models for Indian Context". In: Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers). Ed. by K. Duh, H. Gomez, and S. Bethard. Mexico City, Mexico: Association for Computational Linguistics, 2024.
  36. L. Zhou et al. "General scales unlock ai evaluation with explanatory and predictive power". In: arXiv preprint arXiv:2503.06378 (2025). · arXiv
  37. H. Zhang et al. "A careful examination of large language model performance on grade school arithmetic". In: Advances in Neural Information Processing Systems (2024).
  38. J. Wei et al. "Chain-of-thought prompting elicits reasoning in large language models". In: Advances in neural information processing systems (2022).

Cited by

Sorted by citations

Metadata

Authors
Bean, A. M., Kearns, R. O., Romanou, A., Hafner, F. S., Mayne, H., Batzner, J., Foroutan Eghlidi, N., Schmitz, C., Korgul, K., Batra, H., Deb, O., Beharry, E., Emde, C., Foster, T., Gausen, A., Grandury, M., Han, S., Hofmann, V., Ibrahim, L., Kim, H., Kirk, H. R., Lin, F., Liu, G., Luettgau, L., Magomere, J., Rystrøm, J., Sotnikova, A., Yang, Y., Zhao, Y., Bibi, A., Bosselut, A., Clark, R., Cohan, A., Foerster, J., Gal, Y., Hale, S., Raji, D., Summerfield, C., Torr, P., Ududec, C., Rocher, L., & Mahdi, A.
Affiliations
  • University of Oxford (GB)
  • École Polytechnique Fédérale de Lausanne (CH)
  • Technical University of Munich (DE)
  • École polytechnique fédérale de Lausanne (EPFL)
  • Hertie School (DE)
  • Stanford University (US)
  • Imperial College London (GB)
  • Universidad Politécnica de Madrid (ES)
  • Stanford University / LegalAI
  • Allen Institute for Artificial Intelligence (US)
  • Yale University (US)
  • AI Security Institute
  • Meedan
  • UC Berkeley
  • UK AI Security Institute
ORCID
Venue
Advances in Neural Information Processing Systems 38
Publisher
Neural Information Processing Systems Foundation, Inc. (NeurIPS)
Published
2025
Pages
19868-19949
ISSN
1049-5258 (ISSN-L)
DOI
10.52202/085713-0590
OpenAlex
W4416268504
Cited by
110
References
1
Citations by year
2026: 108 · 2025: 2
FWCI
39.97
Type
conference paper
Language
en
Open access
Locations
  • neural information processing systems (publishedVersion)
  • Oxford University Research Archive (ORA) (University of Oxford) (acceptedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • Infoscience (Ecole Polytechnique Fédérale de Lausanne) (submittedVersion, cc-by-nc-nd)
  • OPUS 4 (Zuse Institute Berlin) (submittedVersion)
  • arXiv (Cornell University) (cc-by)
Funders
  • National Science Foundation (2139841)
  • UK Research and Innovation (EP/S02428X/1, EP/W002981/1, EP/W524311/1, MR/Y015711/1, ES/P000649/1)
  • European Commission (101140087)
  • École Polytechnique Fédérale de Lausanne
  • Schweizerischer Nationalfonds zur Förderung der Wissenschaftlichen Forschung (215390)
  • AXA Research Fund
  • Innosuisse - Schweizerische Agentur für Innovationsförderung (PFFS-21-29)
  • Engineering and Physical Sciences Research Council (EP/W524311/1, EP/S02428X/1, EP/S024050/1, EP/W002981/1)
  • Economic and Social Research Council (EP/W524311/1, ES/P000649/, P000649, ES/P000649/1)
  • Jesus College, University of Cambridge
  • Agencia Estatal de Investigación
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
  • Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
  • Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
Keywords
large language models, benchmark design, construct validity, LLM evaluation, safety, robustness, scoring metrics, expert review

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review