Investigating Data Contamination in Modern Benchmarks for Large Language Models
Open accessAbstract
Recent observations have underscored a disparity between the inflated benchmark scores and the actual performance of LLMs, raising concerns about potential contamination of evaluation benchmarks. This issue is especially critical for closed-source models and certain open-source models where training data transparency is lacking. In this paper we study data contamination by proposing two methods tailored for both open-source and proprietary LLMs. We first introduce a retrieval-based system to explore potential overlaps between evaluation benchmarks and pretraining corpora. We further present a novel investigation protocol named \textbf{T}estset \textbf{S}lot Guessing (\textit{TS-Guessing}), applicable to both open and proprietary models. This approach entails masking a wrong answer in a multiple-choice question and prompting the model to fill in the gap. Additionally, it involves obscuring an unlikely word in an evaluation example and asking the model to produce it. We find that certain commercial LLMs could surprisingly guess the missing option in various test sets. Specifically, in the TruthfulQA benchmark, we find that LLMs exhibit notable performance improvement when provided with additional metadata in the benchmark. Further, in the MMLU benchmark, ChatGPT and GPT-4 demonstrated an exact match rate of 52\% and 57\%, respectively, in guessing the missing options in benchmark test data. We hope these results underscore the need for more robust evaluation methodologies and benchmarks in the field.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
gemma-7b is contaminated on TruthfulQA.
“Qwen2.5-3B is identified as contaminated on Arc-C and MMLU, while gemma-7b is contaminated on TruthfulQA.”
From The Emperor's New Clothes in Benchmarking? A Rigorous Examination of Mitigation Strategies for LLM Benchmark Data Contamination (2025) · cites this paper as Deng et al., 2023
The Metrics section reports significant overlap between TruthfulQA and pre-training corpora, supporting a general concern about contamination. But Table 2 (Model Company section) reports results for other models, and a search for Gemma found no passage identifying gemma-7b as contaminated on TruthfulQA. (LLM, conf=0.84)
Experimental — accuracy not yet measuredMany current coding benchmarks are susceptible to contamination because widely used open-source repositories are prime candidates for inclusion in large-scale web-crawled pre-training corpora.
“First, many benchmarks are susceptible to contamination [7, 13, 15, 20], as exemplified by recent works [5, 7, 15] and social media posts [1, 19]. This risk arises because widely used open-source repositories-particularly those distributed under permissive licenses (e.g., MIT, Apache 2.0, BSD)-are prime candidates for inclusion in the large-scale web-crawled corpora used to pre-train LLMs [3].”
From SWE-Bench Pro: Can AI Agents Solve Long-Horizon Software Engineering Tasks? (2025) · cites this paper as [7]
The Introduction (page unavailable) says that large-scale crawled web training data may include evaluation benchmarks, supporting the general contamination risk. The paper focuses on NLP benchmarks, not coding benchmarks. The repository-and-license explanation appears in a separate sentence cited to [3], so it is not evidence attributable to this source. (LLM, conf=0.86)
Experimental — accuracy not yet measuredThe study applies three BDC detection methods from distinct categories-Min-K% Prob, Sharded Rank Comparison Test, and TS-Guessing-to 14 candidate models, and retains only models deemed uncontaminated by all three detection methods on all benchmarks.
“To achieve this, we apply three BDC detection methods from distinct categories-Min-K% Prob (Shi et al., 2023), Sharded Rank Comparison Test (Oren et al., 2023), and TS-Guessing (Deng et al., 2023)-to 14 candidate models. We adopt a rigorous criterion: only models deemed uncontaminated by all three detection methods on all benchmarks are retained”
From The Emperor's New Clothes in Benchmarking? A Rigorous Examination of Mitigation Strategies for LLM Benchmark Data Contamination (2025) · cites this paper as Deng et al., 2023
A high Rouge-L F1 score between the model's prediction and the ground truth indicates that the model can accurately predict the masked option, suggesting prior exposure to the benchmark data.
“A high Rough-L F1 score between the model's prediction and the ground truth (see Tab. 10) indicates that the model can accurately predict the masked option, suggesting prior exposure to the benchmark data.”
From The Emperor's New Clothes in Benchmarking? A Rigorous Examination of Mitigation Strategies for LLM Benchmark Data Contamination (2025) · cites this paper as Deng et al., 2023
Access to pre-training data enables leakage detection through overlap analysis.
“Access to pre-training data enables leakage detection through overlap analysis, as highlighted in previous research (Dodge et al., 2021;Deng et al., 2023); even without direct access, details like data sources and time frames can reveal leakages (Li, 2023a;b).”
From Benchmarking Benchmark Leakage in Large Language Models (2024) · cites this paper as Deng et al., 2023
Qwen2.5-3B is identified as contaminated on Arc-C and MMLU.
“Qwen2.5-3B is identified as contaminated on Arc-C and MMLU, while gemma-7b is contaminated on TruthfulQA.”
From The Emperor's New Clothes in Benchmarking? A Rigorous Examination of Mitigation Strategies for LLM Benchmark Data Contamination (2025) · cites this paper as Deng et al., 2023
The paper’s §4.2.1 Setup Domains lists the models evaluated, but does not include Qwen2.5-3B. Searches for Qwen2.5-3B with Arc-C and MMLU found no passage identifying that model as contaminated on either benchmark; the paper’s MMLU contamination experiment instead concerns ChatGPT. (LLM, conf=0.78)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
No references could be resolved yet.
Research works printed but not linked: 21
- Stella Rose Biderman, Hailey Schoelkopf, Quentin G. Anthony, Herbie Bradley, Kyle O'Brien, Eric Hal- lahan, Mohammad Aflah Khan, Shivanshu Purohit, USVSN Sai Prashanth, Edward Raff, Aviya Skowron, Lintang Sutawika, and Oskar van der Wal. 2023. Pythia: A Suite for Analyzing Large Language Mod- els Across Training and Scaling. ArXiv preprint, abs/2304.01373. · arXiv
- Yonatan Bisk, Rowan Zellers, Ronan LeBras, Jianfeng Gao, and Yejin Choi. 2020. PIQA: Reasoning about Physical Commonsense in Natural Language. In The Thirty-Fourth AAAI Conference on Artificial Intelli- gence, AAAI 2020, The Thirty-Second Innovative Ap- plications of Artificial Intelligence Conference, IAAI 2020, The Tenth AAAI Symposium on Educational Advances in Artificial Intelligence, EAAI 2020, New York, NY, USA, February 7-12, 2020, pages 7432- 7439. AAAI Press.
- Sidney Black, Stella Biderman, Eric Hallahan, Quentin Anthony, Leo Gao, Laurence Golding, Horace He, Connor Leahy, Kyle McDonell, Jason Phang, Michael Pieler, Usvsn Sai Prashanth, Shivanshu Puro- hit, Laria Reynolds, Jonathan Tow, Ben Wang, and Samuel Weinbach. 2022. GPT-NeoX-20B: An Open- Source Autoregressive Language Model. In Proceed- ings of BigScience Episode #5 -Workshop on Chal- lenges & Perspectives in Creating Large Language Models, pages 95-136, virtual+Dublin. Association for Computational Linguistics. · DOI
- Terra Blevins and Luke Zettlemoyer. 2022. Language Contamination Helps Explains the Cross-lingual Ca- pabilities of English Pretrained Models. In Proceed- ings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 3563-3574, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
- Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. 2020. Language Models are Few-Shot Learners. In Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Pro- cessing Systems 2020, NeurIPS 2020, December 6- 12, 2020, virtual.
- Jesse Dodge, Maarten Sap, Ana Marasović, William Agnew, Gabriel Ilharco, Dirk Groeneveld, Mar- garet Mitchell, and Matt Gardner. 2021. Docu- menting Large Webtext Corpora: A Case Study on the Colossal Clean Crawled Corpus, url = https://aclanthology.org/2021.emnlp-main.98. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 1286-1305, Online and Punta Cana, Dominican Re- public. Association for Computational Linguistics. · DOI · Paper page
- Jinlan Fu, See-Kiong Ng, Zhengbao Jiang, and Pengfei Liu. 2023. GPTScore: Evaluate as You Desire. ArXiv preprint, abs/2302.04166. · arXiv
- Suriya Gunasekar, Yi Zhang, Jyoti Aneja, Caio Ce- sar Teodoro Mendes, Allison Del Giorno, Sivakanth Gopi, Mojan Javaheripi, Piero C. Kauffmann, Gus- tavo de Rosa, Olli Saarikivi, Adil Salim, S. Shah, Harkirat Singh Behl, Xin Wang, Sébastien Bubeck, Ronen Eldan, Adam Tauman Kalai, Yin Tat Lee, and Yuan-Fang Li. 2023. Textbooks Are All You Need. ArXiv preprint, abs/2306.11644. · arXiv
- Adi Haviv, Ido Cohen, Jacob Gidron, Roei Schuster, Yoav Goldberg, and Mor Geva. 2023. Understanding Transformer Memorization Recall Through Idioms. In Proceedings of the 17th Conference of the Euro- pean Chapter of the Association for Computational Linguistics, pages 248-264, Dubrovnik, Croatia. As- sociation for Computational Linguistics.
- Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Stein- hardt. 2021. Measuring Massive Multitask Language Understanding. In 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021. OpenReview.net.
- Chin-Yew Lin. 2004. ROUGE: A Package for Auto- matic Evaluation of Summaries. In Text Summariza- tion Branches Out, pages 74-81, Barcelona, Spain. Association for Computational Linguistics.
- Stephanie Lin, Jacob Hilton, and Owain Evans. 2022. TruthfulQA: Measuring How Models Mimic Human Falsehoods. In Proceedings of the 60th Annual Meet- ing of the Association for Computational Linguistics (Volume 1: Long Papers), pages 3214-3252, Dublin, Ireland. Association for Computational Linguistics. · DOI · Paper page
- Lingjie Mei, Jiayuan Mao, Ziqi Wang, Chuang Gan, and Joshua B. Tenenbaum. 2022. FALCON: Fast Visual Concept Learning by Integrating Images, Linguistic descriptions, and Conceptual Relations. In The Tenth International Conference on Learning Representa- tions, ICLR 2022, Virtual Event, April 25-29, 2022.
- Todor Mihaylov, Peter Clark, Tushar Khot, and Ashish Sabharwal. 2018. Can a Suit of Armor Conduct Elec- tricity? A New Dataset for Open Book Question An- swering. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 2381-2391, Brussels, Belgium. Association for Computational Linguistics. · DOI
- Aleksandra Piktus, Christopher Akiki, Paulo Ville- gas, Hugo Laurençon, Gérard Dupont, Sasha Luc- cioni, Yacine Jernite, and Anna Rogers. 2023a. The ROOTS Search Tool: Data Transparency for LLMs. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 3: System Demonstrations), pages 304-314, Toronto, Canada. Association for Computational Linguistics. · DOI
- Matt Post. 2018. A Call for Clarity in Reporting BLEU Scores. In Proceedings of the Third Conference on Machine Translation: Research Papers, pages 186- 191, Brussels, Belgium. Association for Computa- tional Linguistics. · DOI · Paper page
- Oscar Sainz, Jon Campos, Iker García-Ferrero, Julen Etxaniz, Oier Lopez de Lacalle, and Eneko Agirre. 2023. NLP evaluation in trouble: On the need to mea- sure LLM data contamination for each benchmark. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 10776-10787, Sin- gapore. Association for Computational Linguistics. · DOI · Paper page
- Keisuke Sakaguchi, Ronan Le Bras, Chandra Bhagavat- ula, and Yejin Choi. 2020. WinoGrande: An Adver- sarial Winograd Schema Challenge at Scale. In The Thirty-Fourth AAAI Conference on Artificial Intelli- gence, AAAI 2020, The Thirty-Second Innovative Ap- plications of Artificial Intelligence Conference, IAAI 2020, The Tenth AAAI Symposium on Educational Advances in Artificial Intelligence, EAAI 2020, New York, NY, USA, February 7-12, 2020, pages 8732- 8740. AAAI Press.
- Thibault Sellam, Dipanjan Das, and Ankur Parikh. 2020. BLEURT: Learning Robust Metrics for Text Genera- tion. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 7881-7892, Online. Association for Computational Linguistics. · DOI · Paper page
- Kristina Toutanvoa and Christopher D. Manning. 2000. Enriching the Knowledge Sources Used in a Maxi- mum Entropy Part-of-Speech Tagger. In 2000 Joint SIGDAT Conference on Empirical Methods in Nat- ural Language Processing and Very Large Corpora, pages 63-70, Hong Kong, China. Association for Computational Linguistics. · DOI
- Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi. 2019. HellaSwag: Can a Machine Really Finish Your Sentence? In Proceed- ings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 4791-4800, Flo- rence, Italy. Association for Computational Linguis- tics. · DOI · Paper page
Cited by
Sorted by citations
Hallucination‐Free? Assessing the Reliability of Leading <scp>AI</scp> Legal Research Tools
Magesh, V., Surani, F., Dahl, M., Suzgun, M., Manning, C. D., & Ho, D. E. (2025) · Journal of Empirical Legal Studies · doi:10.1111/jels.12413 · Cited by 182 · see citing works
Natural language processing in the era of large language models
Zubiaga, A. (2024) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2023.1350306 · Cited by 81 · see citing works
Dong, Y., Jiang, X., Liu, H., Jin, Z., Gu, B., Yang, M., & Li, G. (2024) · Findings of the Association for Computational Linguistics ACL 2024 · doi:10.18653/v1/2024.findings-acl.716 · Cited by 52 · see citing works
Task Contamination: Language Models May Not Be Few-Shot Anymore
Li, C., & Flanigan, J. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i16.29808 · Cited by 47 · see citing works
Lampinen, A. (2024) · Computational Linguistics · doi:10.1162/coli_a_00525 · Cited by 32 · see citing works
A Large-Scale Study of Relevance Assessments with Large Language Models Using UMBRELA
Upadhyay, S., Pradeep, R., Thakur, N., Campos, D., Craswell, N., Soboroff, I., & Lin, J. (2025) · Proceedings of the 2025 International ACM SIGIR Conference on Innovative Concepts and Theories in Information Retrieval (ICTIR) · doi:10.1145/3731120.3744605 · Cited by 11 · see citing works
Benchmarking Benchmark Leakage in Large Language Models
Xu, R., Wang, Z., Fan, R.-Z., & Liu, P. (2024) · arXiv · doi:10.48550/arxiv.2404.18824 · Cited by 6 · see citing works
SWE-Bench Pro: Can AI Agents Solve Long-Horizon Software Engineering Tasks?
Deng, X., Da, J., Pan, E., He, Y. Y., Ide, C., Garg, K., Lauffer, N., Park, A., Pasari, N., Rane, C., et al. (2025) · arXiv · doi:10.48550/arxiv.2509.16941 · Cited by 4 · see citing works
“Notably, the state-of-the-art agents have reported over 70% pass rate on SWE-Bench-Verified [15], a subset of SWE-Bench that is verifiably solvable by human programmers.”
Quantification of Large Language Model Distillation
Lee, S., Zhou, J., Ao, C., Li, K., Du, X., He, S., Wu, H., Liu, T., Liu, J., Alinejad-Rokny, H., et al. (2025) · Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2025.acl-long.248 · Cited by 4 · see citing works
Preference Leakage: A Contamination Problem in LLM-as-a-judge
Li, D., Sun, R., Huang, Y., Zhong, M., Jiang, B., Han, J., Zhang, X., Wang, W., & Liu, H. (2025) · arXiv · doi:10.48550/arxiv.2502.01534 · Cited by 2 · see citing works
“• Judge LLM, M J , providing a scoring function S M J ( y | x ) that assesses output y for prompt x .”
Sun, Y., Wang, H., Li, D., Wang, G., & Zhang, H. (2025) · arXiv · doi:10.48550/arxiv.2503.16402
“The Rouge-L F1 Scores of TS-Guessing (Deng et al., 2023), computed for all candidate LLMs across three benchmarks.”
InfluentialFLAWS: A Benchmark for Error Identification and Localization in Scientific Papers
Xi, S., Rao, V., Payan, J., & Shah, N. B. (2025) · arXiv · doi:10.48550/arxiv.2511.21843
A Comment On "The Illusion of Thinking": Reframing the Reasoning Cliff as an Agentic Gap
Khan, S., Madhavan, S., & Natarajan, K. (2025) · arXiv · doi:10.48550/arxiv.2506.18957
- Authors
- Deng, C., Zhao, Y., Tang, X., Gerstein, M., & Cohan, A.
- Affiliations
- Georgia Institute of Technology (US)
- Yale University (US)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2023
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2311.09783
- OpenAlex
- W4388788574
- Cited by
- 8
- References
- 0
- Citations by year
- 2026: 1 · 2025: 3 · 2024: 4
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- arXiv (Cornell University) (submittedVersion, cc-by-nc-nd)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
- Keywords
- contaminated data, benchmark evaluation, TruthfulQA, MMLU, ChatGPT, GPT-4, open-source vs closed-source models
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
