Preference Leakage: A Contamination Problem in LLM-as-a-judge
Open accessAbstract
Large Language Models (LLMs) as judges and LLM-based data synthesis have emerged as two fundamental LLM-driven data annotation methods in model development. While their combination significantly enhances the efficiency of model training and evaluation, little attention has been given to the potential contamination brought by this new model development paradigm. In this work, we expose preference leakage, a contamination problem in LLM-as-a-judge caused by the relatedness between the synthetic data generators and LLM-based evaluators. To study this issue, we first define three common relatednesses between the data generator LLM and the judge LLM: being the same model, having an inheritance relationship, and belonging to the same model family. Through extensive experiments, we empirically confirm the bias of judges towards their related student models caused by preference leakage across multiple LLM baselines and benchmarks. Further analysis suggests that preference leakage is a pervasive and real-world problem that is harder to detect compared to previously identified biases in LLM-as-a-judge scenarios. All of these findings imply that preference leakage is a widespread and challenging problem in the area of LLM-as-a-judge. We release all codes and data at: https://github.com/David-Li0406/Preference-Leakage.
AI snapshot
Written from the paper’s full text.
TL;DR
This paper identifies “preference leakage,” a contamination problem in LLM-as-a-judge where a judge LLM favors student models trained on synthetic data from a related generator LLM. Across multiple benchmarks and model pairs, the authors find systematic bias that is pervasive, scales with synthetic-data proportion and relatedness, and is hard for judges to recognize.
Key findings
- Preference leakage exists in most model pairs: with the same data generator and judge, the average preference leakage score (PLS) was 23.6% (Arena-Hard 28.7%, AlpacaEval 2.0 18.4%).
- Relatedness severity varies: inheritance settings averaged 19.3% with same instructions and 22.3% with different instructions; same-family same-series averaged 8.9%, while different-series averaged 2.8%.
- Learning method matters: SFT showed the highest average leakage at 23.6%, DPO was 5.2%, and ICL was -2.7%. Smaller student models exhibited higher PLS than larger counterparts.
- In real-world leaderboards, preference leakage produced a ranking difference of 1.33, higher than egocentric bias’s 1.00. Judge LLMs performed poorly at recognizing their related student models’ generations (e.g., GPT-4o 41.0% pointwise / 52.0% pairwise; Gemini-1.5 53.2% / 44.2%; LLaMA-3.3 41.8% / 29.8%), while a BERT classifier reached 82.4% accuracy.
Methods
- The authors define three generator–judge relatednesses: being the same model, having an inheritance relationship, and belonging to the same model family. They formalize preference leakage and propose the preference leakage score (PLS) to quantify judge bias toward related student models.
- Main experiments used GPT-4o-2024-11-20, Gemini-1.5-flash, and LLaMA-3.3-70B-Instruct-turbo as data generators/judges; Mistral-7B-v0.1 and Qwen-2.5-14B pretrained models as students; 30,000 prompts sampled from Ultrafeedback for synthetic data; SFT training; and pairwise evaluation on Arena-Hard (500 questions) and AlpacaEval 2.0 (805 questions).
- Further analyses covered data mixing (10%, 30%, 50%, 70% synthetic data), relatedness settings, learning methods (SFT, DPO, ICL), real-world leaderboard impact, judge recognition of student generations, and question-type/judgment-dimension categories. Manual annotation used 100 randomly selected AlpacaEval 2.0 questions with three annotators and majority voting.
Limitations
- The paper states that detection, mitigation, and calibration methods for preference leakage are left for future work.
- Due to cost, several further analyses were conducted only on the Mistral-GPT-4o vs Mistral-Gemini-1.5 pair, and manual annotation covered only 100 randomly selected AlpacaEval 2.0 questions.
- Evident from the text: the study focuses on specific LLM baselines, benchmarks, and training setups, so broader generalization across other judges, data sources, and evaluation protocols is not established.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
0 of 21 cited sources checked · 0 supported · 0 flagged
- Findingown data
Judges show systematic bias towards their related student models due to preference leakage across multiple LLM baselines and benchmarks.
“Through extensive experiments, we empirically confirm the bias of judges towards their related student models caused by preference leakage across multiple LLM baselines and benchmarks.”
- Also stated in: § 1 Introduction · hedged
“The analysis results suggest an obvious bias of judging LLMs toward their related student models due to preference leakage.”
- Also stated in: § 4.2 Main Results · plain
“Preference leakage exists in most model pairs.”
Quote in the paperSubstantial claim - Also stated in: § 1 Introduction · hedged
- Findingown data
In inheritance settings, preference leakage causes obvious bias of judges towards their related students.
“Preference leakage under inheritance settings causes obvious bias of judges towards their related students.”
Quote in the paperSubstantial claim - Findingown data
Models within the same series tend to cause more significant bias.
“Models within the same series tend to cause more significant bias.”
Quote in the paperSubstantial claim - Findingown data
Preference leakage exists in most model pairs in the main SFT experiment.
“Preference leakage exists in most model pairs.”
- Also stated in: § 4.2 Main Results · hedged
“This finding suggests that preference leakage, along with the resulting bias, is widespread in SFT when the data generator and evaluator are the same.”
Quote in the paperSubstantial claim - Also stated in: § 4.2 Main Results · hedged
- Findingown data
Smaller student models cause even more bias from judge LLMs.
“Smaller student models cause even more bias from judge LLMs.”
- Also stated in: § 4.2 Main Results · hedged
“a notable finding is that the smallest models (Qwen-2.5-3Band Qwen-3-1.7B)exhibit the highest PL scores than their larger counterparts, indicating greater bias from preference leakage.”
Quote in the paperSubstantial claim - Also stated in: § 4.2 Main Results · hedged
- Findingown data
PL scores in Arena-Hard are generally higher than those in AlpacaEval 2.0.
“the PL scores in ArenaHard are generally higher than those in AlpacaEval 2.0.”
Quote in the paperNot a substantial claim - Findingown data
A fine-tuned BERT classifier demonstrates high accuracy in classifying responses generated by each student model.
“the fine-tuned BERT classification demonstrates a high accuracy score in classifier responses generated by each student model.”
Quote in the paperSubstantial claim - Findingreasoning
Preference leakage is particularly pervasive in popular LLM-as-a-judge benchmarks and LLM-relevant studies.
“During our preliminary study, we find this issue is particularly pervasive in popular LLM-as-a-judge benchmarks (e.g., AlpacaEval 2.0 [14] and Arena-Hard [39]) and LLM-relevant studies (more details can be found in Appendix A), due to the common reliance on the most advanced LLMs, such as GPT-4 [1], for both data synthesis and evaluation to ensure the highest quality outputs.”
Quote in the paperSubstantial claim0 of 3 sources checked- cites [14]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites [39]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites [1]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Findingown data
Judge LLMs do not show good performance in recognizing the generation of their student models, with accuracy around random guess.
“Judge LLMs do not show good performance in recognizing the generation of their student models.”
- Also stated in: § GPT-4 Preview GPT-4 Preview 1.00 Preference Leakage · plain
“the recognition performance of each judge LLM in the content of related students is poor, with accuracy around the performance of random guess.”
Quote in the paperSubstantial claim - Also stated in: § GPT-4 Preview GPT-4 Preview 1.00 Preference Leakage · plain
- Findingown data
The degree of preference leakage is directly proportional to the amount of synthetic data.
“The degree of preference leakage is directly proportional to the amount of synthetic data.”
- Also stated in: § 5.1 Data Mixing Analysis · boosted
“We observe a strong correlation between the proportion of synthetic data in the mixture and the preference leakage score, with no clear threshold separating cases with preference leakage from those without.”
Quote in the paperSubstantial claim - Also stated in: § 5.1 Data Mixing Analysis · boosted
- Findingown data
Subjective questions and judgment dimensions tend to lead to more bias.
“Subjective question and judgment dimension tend to lead to more bias.”
Quote in the paperNot a substantial claim - Findingnone given
Student model pairs with similar performance or more powerful student models tend to exhibit greater preference leakage.
“While we have concluded that student model pairs with similar performance or more powerful student models tend to exhibit greater preference leakage, we also examine whether different data generator”
Quote in the paperSubstantial claim - Findingown data
Several candidate models distilled from GPT-4 or other GPT-series models (e.g., Vicuna and Alpaca) appear across AlpacaEval 2.0, Arena-Hard, and MTBench.
“Our analysis reveals that several candidate models distilled from GPT-4 or other GPT-series models (e.g., Vicuna and Alpaca) appear across all these leaderboards and benchmarks, suggesting that preference leakage is a pervasive issue in these datasets.”
Quote in the paperSubstantial claim - Findingown data
For Mistral-GPT-4o vs Mistral-Gemini-1.5 on AlpacaEval 2.0, the Gemini-1.5 judge gives Mistral-GPT-4o wins 36.8% and Mistral-Gemini-1.5 wins 63.2%.
“| Gemini-1.5 | 36.8% | | | 63.2% |”
Quote in the paperSubstantial claim - Findingown data
BERT response classification accuracy is 82.4%.
“| RC | BERT | 82.4% | |”
Quote in the paperNot a substantial claim - Interpretationown data
Preference leakage is a contamination problem in LLM-as-a-judge caused by relatedness between synthetic data generators and LLM-based evaluators.
“In this work, we expose preference leakage, a contamination problem in LLM-as-a-judge caused by the relatedness between the synthetic data generators and LLM-based evaluators.”
- Also stated in: § 1 Introduction · plain
“In this work, we aim to highlight a subtle yet critical bias in LLM-as-a-Judge: Preference Leakage.This issue arises when the LLMs used for data generation and evaluation are closely related, causing the preference of the LLM evaluators to leak to the student models through synthetic data and thus inflating the evaluation score (as illustrated in Figure 1).”
- Also stated in: § 3.2 Defining Preference Leakage in LLM-based Evaluation · plain
“Preference leakage occurs when the evaluation score assigned by M J to M S 's outputs is inflated due to an underlying relatedness between M G and M J .”
Quote in the paperSubstantial claim - Also stated in: § 1 Introduction · plain
- Interpretationreasoning
Preference leakage can occur even with a small amount of leaked synthetic data.
“This suggests that preference leakage can occur even with a small amount of leaked synthetic data, posing significant challenges for its detection.”
Quote in the paperSubstantial claim - Premisecitation
Arena-Hard evaluation agreement with Chatbot Arena hard prompts achieved 96.7% Spearman correlation.
“the evaluation agreement between Arena-Hard and Chatbot Arena [79]'s hard prompts achieved a 96.7% Spearman correlation, demonstrating the consistency of Arena-Hard with human preferences [39].”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [39]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Study design and conductown data
The average agreement of three annotators is 78.6.
“We also calculate the average agreement of three annotators and find it to be 78.6,indicating a relatively consistent annotation result.”
Quote in the paperNot a substantial claim - Interpretationown data
Certain characteristics such as style and format are embedded in student models through synthetic responses.
“This suggests that certain characteristics-such as style and format-are embedded in the student models through the synthetic responses.”
Quote in the paperSubstantial claim - Interpretationown data
Preference leakage is a pervasive issue in AlpacaEval 2.0, Arena-Hard, and MTBench.
“Our analysis reveals that several candidate models distilled from GPT-4 or other GPT-series models (e.g., Vicuna and Alpaca) appear across all these leaderboards and benchmarks, suggesting that preference leakage is a pervasive issue in these datasets.”
Quote in the paperSubstantial claim - Premisecitation
A bunch of LLM-relevant research studies utilize the same or related model(s) to do distillation/data synthesis and evaluation.
“Besides, we also examine if preference leakage exists in LLM-relevant research studies and also find a bunch of work utilizing the same or related model(s) to do distillation/ data synthesis and evaluation [70,44,31,38,62,55].”
Quote in the paperNot a substantial claim0 of 6 sources checked- cites [70]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites [44]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites [31]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites [38]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites [62]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites [55]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- May combine two claims. 27
- The quote only partly states this claim. 3
- Classified as probably not a substantial claim. 11
May combine two claims.
Findingown dataGemini-1.5 shows a strong bias toward its students, followed by GPT-4o, with LLaMA-3.3 displaying the least bias.
“we observe that Gemini-1.5 shows a strong bias toward its students, followed by GPT-4o, with LLaMA-3.3 displaying the least bias.”
Quote in the paperSubstantial claimMay combine two claims. The quote only partly states this claim.
Findingown dataIn real-world LLM-as-a-Judge leaderboards, preference leakage causes a ranking difference of 1.33, higher than egocentric bias's 1.00.
“Preference Leakage Vicuna 7B/ 13B/ 33B 1.33”
- Also stated in: § GPT-4 Preview GPT-4 Preview 1.00 Preference Leakage · plain
“the ranking difference associated with preference leakage is even higher than that of egocentric bias, highlighting the substantial impact of preference leakage on real-world LLM-as-a-Judge leaderboards.”
Quote in the paperSubstantial claim- Also stated in: § GPT-4 Preview GPT-4 Preview 1.00 Preference Leakage · plain
May combine two claims. The quote only partly states this claim.
Findingown dataPreference leakage consistently affects judge LLMs under various data mixing strategies, relatedness settings, tuning techniques and real-world applications.
“finding that preference leakage consistently affects judge LLMs.”
- Also stated in: § 6 Conclusion · plain
“preference leakage can cause significant bias across diverse scenarios.”
Quote in the paperSubstantial claim- Also stated in: § 6 Conclusion · plain
May combine two claims.
Findingown dataThe severity of preference leakage correlates with the degree of relatedness between the data generator and LLM judges, as well as the proportion of synthetic data.
“the severity of preference leakage correlates with the degree of relatedness between the data generator and LLM judges, as well as the proportion of synthetic data.”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataFor same family with different series, the preference leakage score is 3.3% on Arena-Hard and 2.2% on AlpacaEval 2.0, with an average of 2.8%.
“| Same Family -w/ different series | 3.3% | 2.2% | 2.8% |”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataFor inheritance with the same instruction, the preference leakage score is 17.8% on Arena-Hard and 20.7% on AlpacaEval 2.0, with an average of 19.3%.
“| Inheritance -w/ same ins. | 17.8% | 20.7% | 19.3% |”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataGPT-4o student recognition accuracy is 41.0% pointwise and 52.0% pairwise.
“| | GPT-4o | 41.0% | 52.0% |”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataWhen the data generator and judge are the same model, the preference leakage score is 28.7% on Arena-Hard and 18.4% on AlpacaEval 2.0, with an average of 23.6%.
“| Same Model | 28.7% | 18.4% | 23.6% |”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataFor Mistral-7B with LLaMA-3.3 as data generator and Gemini-1.5 as judge, the preference leakage score is 13.1% on Arena-Hard and 19.8% on AlpacaEval 2.0, with an average of 16.4%.
“| | LLaMA-3.3 & Gemini-1.5 | 13.1% | 19.8% | 16.4% |”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataFor Qwen-2.5-14B with GPT-4o as data generator and Gemini-1.5 as judge, the preference leakage score is 37.1% on Arena-Hard and 18.6% on AlpacaEval 2.0, with an average of 27.9%.
“| | GPT-4o & Gemini-1.5 | 37.1% | 18.6% | 27.9% |”
Quote in the paperSubstantial claimClassified as probably not a substantial claim.
Findingown dataFor Mistral-GPT-4o vs Mistral-Gemini-1.5 on AlpacaEval 2.0, the GPT-4o judge gives Mistral-GPT-4o wins 55.1% and Mistral-Gemini-1.5 wins 44.9%.
“| GPT-4o | 55.1% | | | 44.9% |”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataFor Qwen-2.5-14B with LLaMA-3.3 as data generator and Gemini-1.5 as judge, the preference leakage score is 25.4% on Arena-Hard and 18.4% on AlpacaEval 2.0, with an average of 21.9%.
“| | LLaMA-3.3 & Gemini-1.5 | 25.4% | 18.4% | 21.9% |”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataLLaMA-3.3 student recognition accuracy is 41.8% pointwise and 29.8% pairwise.
“| | LLaMA-3.3 | 41.8% | 29.8% |”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataWith ICL as the learning method, the preference leakage score is -4.2% on Arena-Hard and -1.1% on AlpacaEval 2.0, with an average of -2.7%.
“| ICL | -4.2% | -1.1% | -2.7% |”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataWith SFT as the learning method, the preference leakage score is 28.7% on Arena-Hard and 18.4% on AlpacaEval 2.0, with an average of 23.6%.
“| SFT | 28.7% | 18.4% | 23.6% |”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataFor same family with the same series, the preference leakage score is 10.1% on Arena-Hard and 7.6% on AlpacaEval 2.0, with an average of 8.9%.
“| Same Family -w/ same series | 10.1% | 7.6% | 8.9% |”
Quote in the paperSubstantial claimMay combine two claims. The quote only partly states this claim.
Findingown dataFor Mistral-7B with GPT-4o as data generator and Gemini-1.5 as judge, the preference leakage score is 28.7% on Arena-Hard and 18.4% on AlpacaEval 2.0, with an average of 23.6%.
“| | GPT-4o & Gemini-1.5 | 28.7% | 18.4% | 23.6% |”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataWith DPO as the learning method, the preference leakage score is 7.7% on Arena-Hard and 2.7% on AlpacaEval 2.0, with an average of 5.2%.
“| DPO | 7.7% | 2.7% | 5.2% |”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataGemini-1.5 student recognition accuracy is 53.2% pointwise and 44.2% pairwise.
“| SR | Gemini-1.5 | 53.2% | 44.2% |”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Findingnone givenDifferent from Arena-Hard, there is no tie in AlpacaEval 2.0.
“Different from Arena-Hard, there is no tie in AlpacaEval 2.0.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Findingown dataThe preference leakage score for Mistral-GPT-4o vs Mistral-Gemini-1.5 on AlpacaEval 2.0 is 18.4%.
“| Preference Leakage Score | | | 18.4% | |”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataFor inheritance with different instructions, the preference leakage score is 18.3% on Arena-Hard and 26.3% on AlpacaEval 2.0, with an average of 22.3%.
“| Inheritance -w/ different ins. | 18.3% | 26.3% | 22.3% |”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataFor Mistral-7B with GPT-4o as data generator and LLaMA-3.3 as judge, the preference leakage score is -1.5% on Arena-Hard and 1.4% on AlpacaEval 2.0, with an average of -0.1%.
“| Mistral-7B | GPT-4o & LLaMA-3.3 | -1.5% | 1.4% | -0.1% |”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataFor Qwen-2.5-14B with GPT-4o as data generator and LLaMA-3.3 as judge, the preference leakage score is 1.0% on Arena-Hard and 2.3% on AlpacaEval 2.0, with an average of 1.7%.
“| Qwen-2.5-14B | GPT-4o & LLaMA-3.3 | 1.0% | 2.3% | 1.7% |”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationAlpacaEval 2.0 significantly reduces the effect of text length on evaluation results compared to version 1.0.
“Compared to version 1.0, AlpacaEval 2.0 significantly reduces the effect of text length on the evaluation results.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [14]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
May combine two claims.
InterpretationcitationPreference leakage can arise from various types of relatedness between data-generating and evaluating LLMs, as well as the intricate flow of synthetic data among modern LLMs.
“It can arise from various types of relatedness between data-generating and evaluating LLMs, as well as the intricate flow of synthetic data among modern LLMs [57].”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [57]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Probably not a substantial claim: may not pass S4 (based).
Study design and conductreasoningUsing pre-trained rather than instruct versions of student models avoids potential preference leakage due to distilling data from other LLMs during instruction-tuning.
“To avoid potential preference leakage due to distilling data from other LLMs during the instruction-tuning process, we choose to use the -PRE-TRAINED version rather than the -INSTRUCT version of these student models.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S4 (based). May combine two claims.
PremisereasoningModels within the same family share a common architectural blueprint and are often developed from foundational models pre-trained on substantially overlapping datasets, leading to correlated preferences and systemic biases.
“Models within such a family typically share a common architectural blueprint (A X ) and are often developed from foundational models pre-trained on substantially overlapping datasets (D X ).This shared foundation (A X , D X ) would lead to correlated preferences and systemic biases characteristic of the common origin:”
Quote in the paperNot a substantial claim- Rests on an assumption: Models within the same family share a common architectural blueprint and are often developed from foundational models pre-trained on substantially overlapping datasets.
May combine two claims.
InterpretationcitationDetecting preference leakage is more challenging because LLMs often do not disclose their training data and distillation quantification is difficult.
“Moreover, detecting preference leakage is also more challenging, given LLMs often do not disclose their training data and the difficulty in distillation quantification [61,32].”
Quote in the paperSubstantial claim0 of 2 sources checked- cites [61]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites [32]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Rests on an assumption: Most LLMs do not disclose their training data.
Probably not a substantial claim: may not pass S3 (load-bearing).
Premisenone givenEgocentric bias is most closely related to preference leakage.
“Among these, egocentric bias, where LLM evaluators tend to favor their generations [29,47,63,68,52,49,5], is most closely related to the preference leakage proposed in this work.”
Quote in the paperNot a substantial claim0 of 7 sources checked- cites [29]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites [47]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites [63]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites [68]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites [52]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites [49]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites [5]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
May combine two claims.
Interpretationown dataPreference leakage is a pervasive and real-world problem that is harder to detect compared to previously identified biases in LLM-as-a-judge scenarios.
“Further analysis suggests that preference leakage is a pervasive and real-world problem that is harder to detect compared to previously identified biases in LLM-as-a-judge scenarios.”
- Also stated in: § 6 Conclusion · boosted
“demonstrating that it is a challenging and hard-to-detect issue, especially in subjective questions and judgment dimensions.”
Quote in the paperSubstantial claim- Rests on an assumption: Most LLMs do not disclose their training data.
- Also stated in: § 6 Conclusion · boosted
Probably not a substantial claim: may not pass S2 (checkable).
InterpretationreasoningPreference leakage presents a more complex and dynamic challenge than egocentric bias.
“in contrast to the relatively straightforward setting of egocentric bias, preference leakage presents a more complex and dynamic challenge.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S4 (based).
Interpretationnone givenIn an inheritance relationship, the descendant model is likely to internalize and thus favor the preferences, styles, or biases of its progenitor.
“In such cases, the descendant model is likely to internalize and thus favor the preferences, styles, or biases of its progenitor.”
Quote in the paperNot a substantial claim- Rests on an assumption: In an inheritance relationship, the descendant model is likely to internalize and favor the preferences, styles, or biases of its progenitor.
May combine two claims.
InterpretationreasoningPreference leakage is a widespread and challenging problem in the area of LLM-as-a-judge.
“All of these findings imply that preference leakage is a widespread and challenging problem in the area of LLM-as-a-judge.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
InterpretationreasoningThe variation in preference leakage across judges may stem from differences in the level of leaked preference in synthetic responses generated by data generator LLMs.
“This variation in preference leakage may stem from differences in the level of leaked preference in the synthetic responses generated by the data generator LLMs.”
Quote in the paperNot a substantial claimMay combine two claims.
Interpretationown dataPreference leakage is a widespread problem in both LLM-as-a-judge datasets and LLM-relevant research.
“All of these suggest preference leakage to be a widespread problem in both LLM-as-a-judge datasets and LLM-relevant research.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
InterpretationreasoningAn LLM with a distinctive style or format offers more opportunities for student models to learn these characteristics, potentially leading to more pronounced preference leakage.
“For instance, an LLM with a distinctive style or format in its responses offers more opportunities for student models to learn these characteristics, potentially leading to more pronounced preference leakage during evaluation.”
Quote in the paperNot a substantial claim
The paper relies on this without support.
The three relatedness types (same model, inheritance, same family) are commonly encountered in real-world applications.
“Each of these scenarios is commonly encountered in real-world applications.”
The paper relies on this without support.
Little attention has been given to potential contamination when generator and evaluator LLMs share a close relationship.
“However, limited attention has been given to the potential contamination that occurs when the generator and evaluator LLMs share a close relationship.”
The paper relies on this without support.
Most LLMs do not disclose their training data.
“especially given that most LLMs do not disclose their training data.”
The paper relies on this without support.
Models within the same family share a common architectural blueprint and are often developed from foundational models pre-trained on substantially overlapping datasets.
“Models within such a family typically share a common architectural blueprint (A X ) and are often developed from foundational models pre-trained on substantially overlapping datasets (D X ).”
The paper relies on this without support.
In an inheritance relationship, the descendant model is likely to internalize and favor the preferences, styles, or biases of its progenitor.
“In such cases, the descendant model is likely to internalize and thus favor the preferences, styles, or biases of its progenitor.”
The paper relies on this without support.
In real-world applications, synthetic data from a single LLM is often mixed with manually-written data or other multi-source synthetic data to train student models.
“In real-world applications, synthetic data from a single LLM is often mixed with manually-written data or other multi-source synthetic data to train student models.”
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Question-type conclusion reverses the reported finding
MinorThe section says objective questions and dimensions show the least preference leakage and subjective ones show more. Its concluding sentence instead says leakage is more significant in objective questions and dimensions.
“This suggests that preference leakage tends to be more significant in objective questions and dimensions, where the contaminated model is more likely to receive biased preference.”
“For question type analysis, we find objective questions with a definitive answer, like mathematical ones, demonstrate the least preference leakage.By contrast, subjective questions that have more than one standard answer, such as programming and writing, usually lead to a more obvious preference leakage.”
Cites
BLEU: a method for automatic evaluation of machine translation
Papineni, K., Roukos, S., Ward, T., & Zhu, W.-J. (2001) · Proceedings of the 40th Annual Meeting on Association for Computational Linguistics - ACL '02 · doi:10.3115/1073083.1073135 · Cited by 21,344 · cited 1× · see citing works
“…from traditional natural language processing tasks [70, 73] to real-world [46, 20], open-ended response generation [65], which greatly limits the applicability of traditional n-gram matching methods (e.g., BLEU [50] and ROUGE [41]) [43, 53] or model-based evaluators [77, 81] for evaluation.”
Cambridge, N. (1977) · doi:10.1177/003591577707000909 · Cited by 5,674
BERTScore: Evaluating Text Generation with BERT
Zhang, T., Kishore, V., Wu, F., Weinberger, K. Q., & Artzi, Y. (2019) · arXiv · doi:10.48550/arxiv.1904.09675 · Cited by 2,865 · cited 1× · see citing works
“To address these challenges, the paradigm of LLM-as-a-judge [79, 34, 24, 82, 37] has been proposed, designed to leverage LLM as evaluators to assess response quality.”
LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language ModelsPreprint
Zheng, Y., Zhang, R., Zhang, J., et al. (2024) · Annual Meeting of the Association for Computational Linguistics · doi:10.48550/arxiv.2403.13372 · Cited by 2,087 · see citing works
Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena
Zheng, L., Chiang, W.-L., Sheng, Y., Zhuang, S., Wu, Z., Zhuang, Y., Lin, Z., Li, Z., Li, D., Xing, E. P., et al. (2023) · arXiv · doi:10.48550/arxiv.2306.05685 · Cited by 1,888 · cited 4× · see citing works
“LLM-as-a-Judge, introduced by Zheng et al. [79], leverages LLMs to automatically evaluate responses and assign rewards.”
InfluentialSelf-citationXun, G.-X., Jia, X.-W., Gopalakrishnan, V., et al. (2017) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2016.2614508 · Cited by 1,767 · cited 1×
“As shown in Table 4, both egocentric bias and preference leakage result in a positive ranking difference, indicating that both lead to evaluator bias favoring the target models.”
WizardLM: Empowering Large Pre-Trained Language Models to Follow Complex InstructionsPreprint
Xu, C., Sun, Q., Zheng, K., Geng, X.-B., Zhao, P., Feng, J., Tao, C., Lin, Q.-W., & Jiang, D. (2023) · International Conference on Learning Representations · doi:10.48550/arxiv.2304.12244 · Cited by 1,362 · cited 1× · see citing works
“The Ultrafeedback dataset includes instructions from several publicly available high-quality datasets such as TruthfulQA [42], FalseQA [19], and Evol-Instruct [66].”
Liu, C., Lowe, R., Serban, I. V., Noseworthy, M., Charlin, L., & Pineau, J. (2016) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d16-1230 · Cited by 1,262 · cited 1× · see citing works
“To address these challenges, the paradigm of LLM-as-a-judge [79, 34, 24, 82, 37] has been proposed, designed to leverage LLM as evaluators to assess response quality.”
TruthfulQA: Measuring How Models Mimic Human Falsehoods
Lin, S., Hilton, J., & Evans, O. (2022) · Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2022.acl-long.229 · Cited by 904 · cited 1× · see citing works
“The Ultrafeedback dataset includes instructions from several publicly available high-quality datasets such as TruthfulQA [42], FalseQA [19], and Evol-Instruct [66].”
AI models collapse when trained on recursively generated data
Shumailov, I., Shumaylov, Z., Zhao, Y., Papernot, N., Anderson, R., & Gal, Y. (2024) · Nature · doi:10.1038/s41586-024-07566-y · Cited by 793 · see citing works
Direct Preference Optimization: Your Language Model is Secretly a Reward Model
Rafailov, R., Sharma, A., Mitchell, E., Manning, C. D., Ermon, S., & Finn, C. (2023) · Advances in Neural Information Processing Systems 36 · doi:10.52202/075280-2338 · Cited by 686 · cited 1× · see citing works
“We also compare three learning methods, supervised fine-tuning (SFT), direct preference optimization (DPO) [51], and in-context learning (ICL) [10], to explore the different influences to them under preference leakage.”
A Survey on In-context Learning
Dong, Q., Li, L., Dai, D., Zheng, C., Ma, J., Li, R., Xia, H., Xu, J., Wu, Z., Chang, B., et al. (2024) · Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2024.emnlp-main.64 · Cited by 649 · see citing works
From Crowdsourced Data to High-Quality Benchmarks: Arena-Hard and BenchBuilder PipelinePreprint
Li, T.-L., Chiang, W.-L., Frick, E., Dunlap, L., Wu, T.-H., Zhu, B.-H., Gonzalez, J. E., & Stoica, I. (2024) · International Conference on Machine Learning · doi:10.48550/arxiv.2406.11939 · Cited by 595 · cited 3× · see citing works
“We choose two representative pairwise evaluation datasets, Arena-Hard [39] and AlpacaEval 2.0 [14], to evaluate the trained student models.”
InfluentialGrattafiori, A., Dubey, A., Jauhri, A., Pandey, A., Kadian, A., Al-Dahle, A., Letman, A., Mathur, A., Schelten, A., Vaughan, A., et al. (2024) · arXiv · doi:10.48550/arxiv.2407.21783 · Cited by 505 · cited 1× · see citing works
“3-70B-Instruct-turbo [13].”
Liu, W., Zeng, W., He, K., et al. (2023) · International Conference on Learning Representations · doi:10.48550/arxiv.2312.15685 · Cited by 456 · cited 1×
“Finally, through recognition and category analyses, we investigate the underlying mechanisms of preference leakage, demonstrating that it is a challenging and hard-to-detect issue, especially in subjective questions and judgment dimensions.”
A Structured Review of the Validity of BLEU
Reiter, E. (2018) · Computational Linguistics · doi:10.1162/coli_a_00322 · Cited by 386 · cited 1× · see citing works
“…from traditional natural language processing tasks [70, 73] to real-world [46, 20], open-ended response generation [65], which greatly limits the applicability of traditional n-gram matching methods (e.g., BLEU [50] and ROUGE [41]) [43, 53] or model-based evaluators [77, 81] for evaluation.”
Jiang, A. Q., Sablayrolles, A., Mensch, A., Bamford, C., Chaplot, D. S., Casas, D. D. L., Bressand, F., Lengyel, G., Lample, G., Saulnier, L., et al. (2023) · arXiv · doi:10.48550/arxiv.2310.06825 · Cited by 340 · cited 1× · see citing works
“1 [23] and Qwen-2.”
Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context
Gemini Team, Georgiev, P., Lei, V. I., Burnell, R., Bai, L., Gulati, A., Tanzer, G., Vincent, D., Pan, Z., Wang, S., et al. (2024) · arXiv · doi:10.48550/arxiv.2403.05530 · Cited by 306 · cited 1× · see citing works
“5-flash [58], and LLaMA-3.”
Self-citationDocumenting Large Webtext Corpora: A Case Study on the Colossal Clean Crawled Corpus
Dodge, J., Sap, M., Marasović, A., Agnew, W., Ilharco, G., Groeneveld, D., Mitchell, M., & Gardner, M. (2021) · Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2021.emnlp-main.98 · Cited by 306 · cited 1× · see citing works
“Golchin and Surdeanu [17] have developed “guided instruction” to flag contaminated instances.”
LIMA: Less Is More for Alignment
Zhou, C., Liu, P., Xu, P., Iyer, S., Sun, J., Mao, Y., Ma, X., Efrat, A., Yu, P., Yu, L., et al. (2023) · Advances in Neural Information Processing Systems 36 · doi:10.52202/075280-2400 · Cited by 275 · see citing works
From Generation to Judgment: Opportunities and Challenges of LLM-as-a-judge
Li, D., Jiang, B., Huang, L., Beigi, A., Zhao, C., Tan, Z., Bhattacharjee, A., Jiang, Y., Chen, C., Wu, T., et al. (2024) · arXiv · doi:10.48550/arxiv.2411.16594 · Cited by 271 · cited 1× · see citing works
“To address these challenges, the paradigm of LLM-as-a-judge [79, 34, 24, 82, 37] has been proposed, designed to leverage LLM as evaluators to assess response quality.”
Self-citationTrustLLM: Trustworthiness in Large Language Models
Huang, Y., Sun, L., Wang, H., Wu, S., Zhang, Q., Li, Y., Gao, C., Huang, Y., Lyu, W., Zhang, Y., et al. (2024) · arXiv · doi:10.48550/arxiv.2401.05561 · Cited by 256 · see citing works
Self-citationOpenAssistant Conversations - Democratizing Large Language Model Alignment
Köpf, A., Kilcher, Y., von Rütte, D., Anagnostidis, S., Tam, Z.-R., Stevens, K., Barhoum, A., Duc, N. M., Stanley, O., Nagyfi, R., et al. (2023) · arXiv · doi:10.48550/arxiv.2304.07327 · Cited by 218 · see citing works
Large Language Models for Data Annotation and Synthesis: A Survey
Tan, Z., Li, D., Wang, S., Beigi, A., Jiang, B., Bhattacharjee, A., Karami, M., Li, J., Cheng, L., & Liu, H. (2024) · Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2024.emnlp-main.54 · Cited by 178 · cited 2× · see citing works
“It can arise from various types of relatedness between data-generating and evaluating LLMs, as well as the intricate flow of synthetic data among modern LLMs [57].”
Self-citationTask Contamination: Language Models May Not Be Few-Shot AnymorePreprint
Li, C., & Flanigan, J. (2023) · AAAI Conference on Artificial Intelligence · doi:10.48550/arxiv.2312.16337 · Cited by 155 · cited 1× · see citing works
“3 Preference Leakage”
Bias and Unfairness in Information Retrieval Systems: New Challenges in the LLM Era
Dai, S., Chen, X., Xu, S., Pang, L., Dong, Z., & Xu, J. (2024) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3637528.3671458 · Cited by 148 · see citing works
Leakage in data mining: formulation, detection, and avoidance
Kaufman, S., Rosset, S., & Perlich, C. (2011) · Proceedings of the 17th ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/2020408.2020496 · Cited by 137 · cited 1× · see citing works
“1 [23] and Qwen-2.”
Qwen, :, Yang, A., Yang, B., Zhang, B., Hui, B., Zheng, B., Yu, B., Li, C., Liu, D., et al. (2024) · arXiv · doi:10.48550/arxiv.2412.15115 · Cited by 128 · cited 1× · see citing works
“5-14B [69].”
Towards a Unified Multi-Dimensional Evaluator for Text Generation
Zhong, M., Liu, Y., Yin, D., Mao, Y., Jiao, Y., Liu, P., Zhu, C., Ji, H., & Han, J. (2022) · Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2022.emnlp-main.131 · Cited by 117 · cited 1× · see citing works
“…from traditional natural language processing tasks [70, 73] to real-world [46, 20], open-ended response generation [65], which greatly limits the applicability of traditional n-gram matching methods (e.g., BLEU [50] and ROUGE [41]) [43, 53] or model-based evaluators [77, 81] for evaluation.”
Self-citationCan LLMs Learn from Previous Mistakes? Investigating LLMs' Errors to Boost for ReasoningPreprint
Tong, Y.-Q., Li, D., Wang, S.-Z., et al. (2024) · Annual Meeting of the Association for Computational Linguistics · doi:10.48550/arxiv.2403.20046 · Cited by 105 · cited 1× · see citing works
“Recent advancements in Large Language Models (LLMs) [1, 22, 60, 74] have empowered various downstream tasks and applications.”
Self-citationSelective Reflection-Tuning: Student-Selected Data Recycling for LLM Instruction-TuningPreprint
Li, M., Chen, L., Chen, J.-H., et al. (2024) · Annual Meeting of the Association for Computational Linguistics · doi:10.48550/arxiv.2402.10110 · Cited by 93 · cited 1× · see citing works
“The table below presents the pointwise and pairwise prompts we use for the recognition analysis.”
RLAIF vs. RLHF: Scaling Reinforcement Learning from Human Feedback with AI Feedback
Lee, H., Phatale, S., Mansoor, H., Mesnard, T., Ferret, J., Lu, K., Bishop, C., Hall, E., Carbune, V., Rastogi, A., et al. (2023) · arXiv · doi:10.48550/arxiv.2309.00267 · Cited by 87 · cited 1× · see citing works
“After providing your explanation, output your final verdict by strictly following this format: "”
Li, D., Yang, S., Tan, Z., et al. (2024) · Conference on Empirical Methods in Natural Language Processing · doi:10.48550/arxiv.2405.04819 · Cited by 86 · see citing works
Self-citationAlignBench: Benchmarking Chinese Alignment of Large Language ModelsPreprint
Liu, X., Lei, X., Wang, S., Huang, Y., Feng, A., Wen, B.-S., Cheng, J., Ke, P., Xu, Y.-F., Tam, W., et al. (2023) · Annual Meeting of the Association for Computational Linguistics · doi:10.48550/arxiv.2311.18743 · Cited by 85 · cited 2× · see citing works
“This approach has gained widespread adoption in areas such as model alignment [78] and benchmarking [45, 75, 16, 82], driving significant progress in the field.”
Self-citationAgentBench: Evaluating LLMs as Agents
Liu, X., Yu, H., Zhang, H., Xu, Y., Lei, X., Lai, H., Gu, Y., Ding, H., Men, K., Yang, K., et al. (2023) · arXiv · doi:10.48550/arxiv.2308.03688 · Cited by 79 · cited 1× · see citing works
“Representatively, LLM-based AI agents’ focus transfer from traditional natural language processing tasks [70, 73] to real-world [46, 20], open-ended response generation [65], which greatly limits the applicability of traditional n-gram matching methods (e.g., BLEU [50] and ROUGE [41]) [43, 53] or…”
Humans or LLMs as the Judge? A Study on Judgement Bias
Chen, G. H., Chen, S., Liu, Z., Jiang, F., & Wang, B. (2024) · Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2024.emnlp-main.474 · Cited by 76 · cited 1× · see citing works
“Despite its promising potential, recent studies have highlighted the vulnerabilities and biases of LLM-as-a-Judge [79, 72, 28, 4, 79, 21, 59, 54].”
Memorization vs. Generalization : Quantifying Data Leakage in NLP Performance Evaluation
Elangovan, A., He, J., & Verspoor, K. (2021) · Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume · doi:10.18653/v1/2021.eacl-main.113 · Cited by 75 · cited 1× · see citing works
“To avoid potential preference leakage due to distilling data from other LLMs during the instruction-tuning process, we choose to use the - PRE - TRAINED version rather than the - INSTRUCT version of these student models.”
Benchmarking Cognitive Biases in Large Language Models as Evaluators
Koo, R., Lee, M., Raheja, V., Park, J. I., Kim, Z. M., & Kang, D. (2023) · arXiv · doi:10.48550/arxiv.2309.17012 · Cited by 72 · cited 1× · see citing works
“Despite its promising potential, recent studies have highlighted the vulnerabilities and biases of LLM-as-a-Judge [79, 72, 28, 4, 79, 21, 59, 54].”
Optimization-based Prompt Injection Attack to LLM-as-a-Judge
Shi, J., Yuan, Z., Liu, Y., Huang, Y., Zhou, P., Sun, L., & Gong, N. Z. (2024) · Proceedings of the 2024 on ACM SIGSAC Conference on Computer and Communications Security · doi:10.1145/3658644.3690291 · Cited by 70 · cited 2× · see citing works
“However, in contrast to the relatively straightforward setting of egocentric bias, preference leakage presents a more complex and dynamic challenge.”
Self-citationBenchmarking Cognitive Biases in Large Language Models as Evaluators
Koo, R., Lee, M., Raheja, V., Park, J. I., Kim, Z. M., & Kang, D. (2024) · Findings of the Association for Computational Linguistics ACL 2024 · doi:10.18653/v1/2024.findings-acl.29 · Cited by 70 · see citing works
Won’t Get Fooled Again: Answering Questions with False PremisesPreprint
Hu, S.-D., Luo, Y., Wang, H., et al. (2023) · Annual Meeting of the Association for Computational Linguistics · doi:10.48550/arxiv.2307.02394 · Cited by 61 · cited 1× · see citing works
“One possible explanation is the difference in question difficulty between the two benchmarks, as ArenaHard contains more challenging questions.”
Exploring Large Language Models for Feature Selection: A Data-centric Perspective
Li, D., Tan, Z., & Liu, H. (2024) · SIGKDD Explorations · doi:10.1145/3715073.3715077 · Cited by 55 · cited 1×
“To address these challenges, the paradigm of LLM-as-a-judge [79, 34, 24, 82, 37] has been proposed, designed to leverage LLM as evaluators to assess response quality.”
Self-citationDong, Y., Jiang, X., Liu, H., Jin, Z., Gu, B., Yang, M., & Li, G. (2024) · arXiv · doi:10.48550/arxiv.2402.15938 · Cited by 52 · cited 1×
“Dong et al. [11] proposed the CDD method to identify peaks in the output distribution to detect data contamination.”
Self-citationDeng, C., Zhao, Y., Heng, Y., et al. (2024) · Annual Meeting of the Association for Computational Linguistics · doi:10.48550/arxiv.2406.14644 · Cited by 51 · cited 2× · see citing works
“Dong et al. [11] proposed the CDD method to identify peaks in the output distribution to detect data contamination.”
OpenAI, :, Jaech, A., Kalai, A., Lerer, A., Richardson, A., El-Kishky, A., Low, A., Helyar, A., Madry, A., et al. (2024) · arXiv · Cited by 50 · cited 1× · see citing works
“Recent advancements in Large Language Models (LLMs) [1, 22, 60, 74] have empowered various downstream tasks and applications.”
Do LLM Evaluators Prefer Themselves for a Reason?Preprint
Chen, W.-L., Wei, Z., Zhu, X.-Y., et al. (2025) · arXiv.org · doi:10.48550/arxiv.2504.03846 · Cited by 49 · cited 1× · see citing works
“It can arise from various types of relatedness between data-generating and evaluating LLMs, as well as the intricate flow of synthetic data among modern LLMs [57].”
Interpreting Pretrained Language Models via Concept Bottlenecks
Tan, Z., Cheng, L., Wang, S., et al. (2023) · Pacific-Asia Conference on Knowledge Discovery and Data Mining · doi:10.48550/arxiv.2311.05014 · Cited by 47
Self-citationInvestigating Data Contamination in Modern Benchmarks for Large Language Models
Deng, C., Zhao, Y., Tang, X., Gerstein, M., & Cohan, A. (2024) · Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers) · doi:10.18653/v1/2024.naacl-long.482 · Cited by 45 · see citing works
Human-like Summarization Evaluation with ChatGPT
Gao, M., Ruan, J., Sun, R., Yin, X., Yang, S., & Wan, X. (2023) · arXiv · doi:10.48550/arxiv.2304.02554 · Cited by 40 · cited 3× · see citing works
“…AI agents’ focus transfer from traditional natural language processing tasks [70, 73] to real-world [46, 20], open-ended response generation [65], which greatly limits the applicability of traditional n-gram matching methods (e.g., BLEU [50] and ROUGE [41]) [43, 53] or model-based evaluators…”
Self-citationContextualization Distillation from Large Language Model for Knowledge Graph CompletionPreprint
Li, D., Tan, Z., Chen, T.-L., et al. (2024) · Findings · doi:10.48550/arxiv.2402.01729 · Cited by 40 · cited 1× · see citing works
“Synthetic data generated by LLMs [15, 57, 35, 36] has become a cornerstone of model training [32].”
Self-citationData Contamination Can Cross Language BarriersPreprint
Yao, F., Zhuang, Y.-F., Sun, Z.-H., et al. (2024) · Conference on Empirical Methods in Natural Language Processing · doi:10.48550/arxiv.2406.13236 · Cited by 37 · cited 1× · see citing works
“Several studies analyze data leakage for specific LLMs [2, 67] and report contamination such as cross-language contamination [71] and task contamination [33] that can evade traditional detection methods.”
Pride and Prejudice: LLM Amplifies Self-Bias in Self-Refinement
Xu, W., Zhu, G., Zhao, X., Pan, L., Li, L., & Wang, W. (2024) · Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2024.acl-long.826 · Cited by 34 · cited 1× · see citing works
“It can arise from various types of relatedness between data-generating and evaluating LLMs, as well as the intricate flow of synthetic data among modern LLMs [57].”
Self-citationTime Travel in LLMs: Tracing Data Contamination in Large Language Models
Golchin, S., & Surdeanu, M. (2023) · arXiv · doi:10.48550/arxiv.2308.08493 · Cited by 29 · cited 1× · see citing works
“Golchin and Surdeanu [17] have developed “guided instruction” to flag contaminated instances.”
Ziya2: Data-centric Learning is All LLMs NeedPreprint
Gan, R., Wu, Z.-W., Sun, R., et al. (2023) · arXiv.org · doi:10.48550/arxiv.2311.03301 · Cited by 27 · cited 1× · see citing works
“Synthetic data generated by LLMs [15, 57, 35, 36] has become a cornerstone of model training [32].”
Self-citationMOSS: An Open Conversational Large Language Model
Sun, T., Zhang, X., He, Z., Li, P., Cheng, Q., Liu, X., Yan, H., Shao, Y., Tang, Q., Zhang, S., et al. (2024) · Machine Intelligence Research · doi:10.1007/s11633-024-1502-8 · Cited by 26 · cited 1× · see citing works
“We first build a data pool based on human-written instruction-tuning data from OASST [30], LIMA [83], and MOSS [56] to supervised fine-tune the pre-trained model.”
Self-citationUltraFeedback: Boosting Language Models with Scaled AI Feedback
Cui, G., Yuan, L., Ding, N., Yao, G., He, B., Zhu, W., Ni, Y., Xie, G., Xie, R., Lin, Y., et al. (2023) · arXiv · doi:10.48550/arxiv.2310.01377 · Cited by 23 · cited 2× · see citing works
“3 Preference Leakage”
InfluentialPosition: Adversarial ML for LLMs Is Not Making Any ProgressPreprint
Rando, J., Zhang, J., Carlini, N., et al. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2502.02260 · Cited by 23 · cited 1× · see citing works
“Among these, egocentric bias, where LLM evaluators tend to favor their generations [29, 47, 63, 68, 52, 49, 5], is most closely related to the preference leakage proposed in this work.”
Can LLMs Improve Multimodal Fact-Checking by Asking Relevant Questions?
Beigi, A., Jiang, B., Li, D., et al. (2024) · BigData Congress [Services Society] · doi:10.1109/bigdata66926.2025.11400846 · Cited by 23
Self-citationSelf-Alignment for Factuality: Mitigating Hallucinations in LLMs via Self-Evaluation
Zhang, X., Peng, B., Tian, Y. Z., Zhou, J., Jin, L., Song, L., Mi, H., & Meng, H. M. L. (2024) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2024.acl-long.107 · Cited by 22 · see citing works
Self-citationBenchmark Data Contamination of Large Language Models: A Survey
Xu, C., Guan, S., Greene, D., & Kechadi, M.-T. (2024) · arXiv · doi:10.48550/arxiv.2406.04244 · Cited by 22 · cited 1× · see citing works
“Several studies analyze data leakage for specific LLMs [2, 67] and report contamination such as cross-language contamination [71] and task contamination [33] that can evade traditional detection methods.”
BPO: Towards Balanced Preference Optimization between Knowledge Breadth and Depth in AlignmentPreprint
Wang, S., Tong, Y.-Q., Zhang, H.-Y., et al. (2024) · North American Chapter of the Association for Computational Linguistics · doi:10.48550/arxiv.2411.10914 · Cited by 22 · see citing works
Self-citationLeak, Cheat, Repeat: Data Contamination and Evaluation Malpractices in Closed-Source LLMs
Balloccu, S., Schmidtová, P., Lango, M., & Dušek, O. (2024) · arXiv · doi:10.48550/arxiv.2402.03927 · Cited by 20 · cited 1× · see citing works
“Several studies analyze data leakage for specific LLMs [2, 67] and report contamination such as cross-language contamination [71] and task contamination [33] that can evade traditional detection methods.”
LiveBench: A Challenging, Contamination-Limited LLM Benchmark
White, C., Dooley, S., Roberts, M., Pal, A., Feuer, B., Jain, S., Shwartz-Ziv, R., Jain, N., Saifullah, K., Dey, S., et al. (2024) · arXiv · doi:10.48550/arxiv.2406.19314 · Cited by 20 · cited 1× · see citing works
“White et al. [64] use the most recent information to update the problem.”
UniGen: A Unified Framework for Textual Dataset Generation Using Large Language ModelsPreprint
Wu, S.-Y., Huang, Y., Gao, C.-J., et al. (2024) · arXiv.org · doi:10.48550/arxiv.2406.18966 · Cited by 20 · cited 1× · see citing works
“…AI agents’ focus transfer from traditional natural language processing tasks [70, 73] to real-world [46, 20], open-ended response generation [65], which greatly limits the applicability of traditional n-gram matching methods (e.g., BLEU [50] and ROUGE [41]) [43, 53] or model-based…”
Self-citationLLM-as-a-Coauthor: Can Mixed Human-Written and Machine-Generated Text Be Detected?
Zhang, Q., Gao, C., Chen, D., Huang, Y., Huang, Y., Sun, Z., Zhang, S., Li, W., Fu, Z., Wan, Y., et al. (2024) · Findings of the Association for Computational Linguistics: NAACL 2024 · doi:10.18653/v1/2024.findings-naacl.29 · Cited by 18 · see citing works
Self-citationLLM Evaluators Recognize and Favor Their Own Generations
Panickssery, A., Bowman, S. R., & Feng, S. (2024) · arXiv · doi:10.48550/arxiv.2404.13076 · Cited by 18 · cited 5× · see citing works
“Among these, egocentric bias, where LLM evaluators tend to favor their generations [29, 47, 63, 68, 52, 49, 5], is most closely related to the preference leakage proposed in this work.”
InfluentialZhang, H., Li, D., Li, Y.-R., Shang, C., Shi, C., & Jiang, Y. (2023) · Workshop on Innovative Use of NLP for Building Educational Applications · doi:10.48550/arxiv.2306.06058 · Cited by 18 · see citing works
Self-citationLength-Controlled AlpacaEval: A Simple Way to Debias Automatic Evaluators
Dubois, Y., Galambosi, B., Liang, P., & Hashimoto, T. B. (2024) · arXiv · doi:10.48550/arxiv.2404.04475 · Cited by 17 · cited 2× · see citing works
“During our preliminary study, we find this issue is particularly pervasive in popular LLM-as-a-judge benchmarks (e.g., AlpacaEval 2.0 [14] and Arena-Hard [39]) and LLM-relevant studies (more details can be found in Appendix A), due to the common reliance on the most advanced LLMs, such as GPT-4…”
InfluentialMeasuring memorization in RLHF for code completion
Pappu, A., Porter, B., Shumailov, I., et al. (2024) · International Conference on Learning Representations · doi:10.48550/arxiv.2406.11715 · Cited by 16
Zhang, H.-Y., Wu, Y.-R., Li, D., et al. (2024) · Annual Meeting of the Association for Computational Linguistics · doi:10.48550/arxiv.2404.10306 · Cited by 16 · see citing works
Self-citationToolPRMBench: Evaluating and Advancing Process Reward Models for Tool-using Agents
Li, D.-W., Yao, Y.-G., Tan, Z., Liu, H., & Guo, R.-C. (2026) · Annual Meeting of the Association for Computational Linguistics · doi:10.48550/arxiv.2601.12294 · Cited by 16
Self-citationPrometheus: Inducing Fine-grained Evaluation Capability in Language Models
Kim, S., Shin, J., Cho, Y., Jang, J., Longpre, S., Lee, H., Yun, S., Shin, S., Kim, S., Thorne, J., et al. (2023) · arXiv · doi:10.48550/arxiv.2310.08491 · Cited by 15 · cited 1× · see citing works
“Additionally, Prometheus, a series of open-source LLMs tailored for LLM-as-a-Judge [26, 27], addresses the prohibitive costs associated with proprietary models, further democratizing the technology.”
LLMs as Narcissistic Evaluators: When Ego Inflates Evaluation Scores
Liu, Y., Moosavi, N. S., & Lin, C. (2023) · arXiv · doi:10.48550/arxiv.2311.09766 · Cited by 13 · see citing works
Law of the Weakest Link: Cross Capabilities of Large Language ModelsPreprint
Zhong, M., Zhang, A., Wang, X.-W., et al. (2024) · arXiv.org · doi:10.48550/arxiv.2409.19951 · Cited by 13 · cited 2× · see citing works
“This approach has gained widespread adoption in areas such as model alignment [78] and benchmarking [45, 75, 16, 82], driving significant progress in the field.”
Self-citationA New Dataset and Empirical Study for Sentence Simplification in Chinese
Yang, S., Sun, R., & Wan, X.-Y. (2023) · Annual Meeting of the Association for Computational Linguistics · doi:10.48550/arxiv.2306.04188 · Cited by 13
Self-citationDuan, S., Khona, M., Iyer, A., et al. (2024) · arXiv.org · doi:10.48550/arxiv.2406.14549 · Cited by 12 · cited 1× · see citing works
“We assume that this gap arises from the differing learning capabilities and behaviors of large and small LLMs: while larger models are more prone to memorizing [12] information that exacerbates data contamination.”
LLMs as Narcissistic Evaluators: When Ego Inflates Evaluation Scores
Liu, Y., Moosavi, N., & Lin, C. (2024) · Findings of the Association for Computational Linguistics ACL 2024 · doi:10.18653/v1/2024.findings-acl.753 · Cited by 12 · see citing works
Justice or Prejudice? Quantifying Biases in LLM-as-a-Judge
Ye, J., Wang, Y., Huang, Y., Chen, D., Zhang, Q., Moniz, N., Gao, T., Geyer, W., Huang, C., Chen, P.-Y., et al. (2024) · arXiv · doi:10.48550/arxiv.2410.02736 · Cited by 11 · cited 3× · see citing works
“For instance, Ye et al. [72] explores various biases and vulnerabilities of LLM-as-a-judge, highlighting the importance of developing a reliable and fair LLM-based evaluation system.”
Self-citationWho Taught You That? Tracing Teachers in Model DistillationPreprint
Wadhwa, S., Shaib, C., Amir, S., et al. (2025) · Annual Meeting of the Association for Computational Linguistics · doi:10.48550/arxiv.2502.06659 · Cited by 10 · cited 1× · see citing works
“Moreover, detecting preference leakage is also more challenging, given LLMs often do not disclose their training data and the difficulty in distillation quantification [61, 32].”
Who's Your Judge? On the Detectability of LLM-Generated Judgments
Li, D.-W., Tan, Z., Zhao, C., Jiang, B., Huang, B.-X., Ma, P., Alnaibari, A., Shu, K., & Liu, H. (2025) · arXiv.org · doi:10.48550/arxiv.2509.25154 · Cited by 9
Self-citationHumans or LLMs as the Judge? A Study on Judgement Biases
Chen, G. H., Chen, S., Liu, Z., Jiang, F., & Wang, B. (2024) · arXiv · doi:10.48550/arxiv.2402.10669 · Cited by 9 · see citing works
Judging the Judges: Evaluating Alignment and Vulnerabilities in LLMs-as-Judges
Thakur, A. S., Choudhary, K., Ramayapally, V. S., Vaidyanathan, S., & Hupkes, D. (2024) · arXiv · doi:10.48550/arxiv.2406.12624 · Cited by 8 · cited 1× · see citing works
“Despite its promising potential, recent studies have highlighted the vulnerabilities and biases of LLM-as-a-Judge [79, 72, 28, 4, 79, 21, 59, 54].”
Investigating Data Contamination in Modern Benchmarks for Large Language Models
Deng, C., Zhao, Y., Tang, X., Gerstein, M., & Cohan, A. (2023) · arXiv · doi:10.48550/arxiv.2311.09783 · Cited by 8 · cited 1× · see citing works
“• Judge LLM, M J , providing a scoring function S M J ( y | x ) that assesses output y for prompt x .”
ShifCon: Enhancing Non-Dominant Language Capabilities with a Shift-based Contrastive FrameworkPreprint
Zhang, H.-Y., Shang, C., Wang, S., et al. (2024) · arXiv.org · doi:10.48550/arxiv.2410.19453 · Cited by 8 · cited 1× · see citing works
“Recent advancements in Large Language Models (LLMs) [1, 22, 60, 74] have empowered various downstream tasks and applications.”
Self-citationPrometheus 2: An Open Source Language Model Specialized in Evaluating Other Language Models
Kim, S., Suk, J., Longpre, S., Lin, B. Y., Shin, J., Welleck, S., Neubig, G., Lee, M., Lee, K., & Seo, M. (2024) · arXiv · doi:10.48550/arxiv.2405.01535 · Cited by 7 · see citing works
MetaTool Benchmark for Large Language Models: Deciding Whether to Use Tools and Which to Use
Huang, Y., Shi, J., Li, Y., Fan, C., Wu, S., Zhang, Q., Liu, Y., Zhou, P., Wan, Y., Gong, N. Z., et al. (2023) · arXiv · doi:10.48550/arxiv.2310.03128 · Cited by 7 · cited 1× · see citing works
“Representatively, LLM-based AI agents’ focus transfer from traditional natural language processing tasks [70, 73] to real-world [46, 20], open-ended response generation [65], which greatly limits the applicability of traditional n-gram matching methods (e.g., BLEU [50] and ROUGE [41]) [43, 53] or…”
Self-citationInvestigating Data Contamination for Pre-training Language ModelsPreprint
Jiang, M., Liu, K. Z., Zhong, M., Schaeffer, R., Ouyang, S., Han, J., & Koyejo, S. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2401.06059 · Cited by 7 · cited 1× · see citing works
“This phenomenon, known as data leakage, can artificially improve the performance of LLMs and undermine the reliability of the assessment [7, 25].”
Self-citationEfficient MAP Estimation of LLM Judgment Performance with Prior Transfer
Qu, H., Choi, I., Tan, Z., Wang, S., Yun, S., Long, Q., Siddiqui, F., Lee, K., & Chen, T.-L. (2025) · arXiv.org · doi:10.48550/arxiv.2504.12589 · Cited by 6 · cited 1×
“However, recent studies point out some weaknesses of such an assessment.”
Self-Preference Bias in LLM-as-a-Judge
Wataoka, K., Takahashi, T., & Ri, R. (2024) · arXiv · doi:10.48550/arxiv.2410.21819 · Cited by 5 · cited 1× · see citing works
“Among these, egocentric bias, where LLM evaluators tend to favor their generations [29, 47, 63, 68, 52, 49, 5], is most closely related to the preference leakage proposed in this work.”
Agent-as-a-Judge: Evaluate Agents with Agents
Zhuge, M., Zhao, C., Ashley, D., Wang, W., Khizbullin, D., Xiong, Y., Liu, Z., Chang, E., Krishnamoorthi, R., Tian, Y., et al. (2024) · arXiv · doi:10.48550/arxiv.2410.10934 · Cited by 5 · cited 1×
“Building on this concept, Zhuge et al. [84] proposed Agent-as-a-Judge, where agentic systems are employed to evaluate other agentic systems.”
Self-citationDistillation Quantification for Large Language ModelsPreprint
Lee, S., Zhou, J., Ao, C., et al. (2025) · arXiv.org · doi:10.48550/arxiv.2501.12619 · Cited by 5 · cited 2× · see citing works
“Moreover, detecting preference leakage is also more challenging, given LLMs often do not disclose their training data and the difficulty in distillation quantification [61, 32].”
A New Dataset and Empirical Study for Sentence Simplification in Chinese
Yang, S., Sun, R., & Wan, X. (2023) · Annual Meeting of the Association for Computational Linguistics (ACL) · doi:10.18653/v1/2023.acl-long.462 · Cited by 5 · see citing works
Self-citationAssessing the Impact of Conspiracy Theories Using Large Language ModelsPreprint
Jiang, B., Li, D., Tan, Z., et al. (2024) · arXiv.org · doi:10.48550/arxiv.2412.07019 · Cited by 4 · cited 1× · see citing works
“To address these challenges, the paradigm of LLM-as-a-judge [79, 34, 24, 82, 37] has been proposed, designed to leverage LLM as evaluators to assess response quality.”
Self-citationSun, R., Liu, M.-Y., Yang, S., et al. (2024) · arXiv.org · doi:10.48550/arxiv.2408.09330 · Cited by 4 · cited 1× · see citing works
“In the future, we aim to explore methods for detecting, preventing, and mitigating this evolving challenge in LLM-as-a-judge systems.”
Self-citationSelf-Alignment for Factuality: Mitigating Hallucinations in LLMs via Self-EvaluationPreprint
Zhang, X., Peng, B., Tian, Y., Zhou, J., Jin, L., Song, L., Mi, H., & Meng, H. M. L. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2402.09267 · Cited by 2 · cited 1× · see citing works
“This approach has gained widespread adoption in areas such as model alignment [78] and benchmarking [45, 75, 16, 82], driving significant progress in the field.”
Self-citationMixEval: Deriving Wisdom of the Crowd from LLM Benchmark Mixtures
Ni, J., Xue, F., Yue, X., Deng, Y., Shah, M., Jain, K., Neubig, G., & You, Y. (2024) · arXiv · doi:10.48550/arxiv.2406.06565 · Cited by 2 · cited 1× · see citing works
“To address data contamination issues, Ni et al. [48] have used web user query detection and benchmark mixture.”
Elephants Never Forget: Testing Language Models for Memorization of Tabular DataPreprint
Bordt, S., Nori, H., & Caruana, R. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2403.06644 · Cited by 2 · cited 1× · see citing works
“This trend contrasts with the influence of model size in data contamination, where larger models are typically more susceptible [3].”
A Survey of Agent Memory in the Second Half: Towards Self-Evolving and Long-Horizon Agents
Huang, W.-C., Zhang, W., Liang, Y., Bei, Y., Chen, Y., Feng, T., Pan, X., Tan, Z., Wang, Y., Wei, T., et al. (2026) · arXiv · doi:10.48550/arxiv.2602.06052 · cited 1×
“However, this also poses substantial challenges to the automatic evaluation of these models.”
Self-citationOpenAI, Achiam, J., Adler, S., Agarwal, S., Ahmad, L., Akkaya, I., Aleman, F. L., Almeida, D., Altenschmidt, J., Altman, S., et al. (2023) · arXiv · doi:10.48550/arxiv.2303.08774 · cited 3×
“Recent advancements in Large Language Models (LLMs) [1, 22, 60, 74] have empowered various downstream tasks and applications.”
InfluentialSelf-citationOpenAssistant Conversations - Democratizing Large Language Model Alignment
Köpf, A., Kilcher, Y., Rütte, D. V., et al. (2023) · Advances in Neural Information Processing Systems 36 · doi:10.52202/075280-2064
Correction to: MOSS: An Open Conversational Large Language Model
Sun, T., Zhang, X., He, Z., et al. (2024) · Machine Intelligence Research · doi:10.1007/s11633-024-1527-z
Self-citation
Flagged first, then by citations
Research works printed but not linked: 1
- Chin-Yew Lin. Rouge: A package for automatic evaluation of summaries. In Text summarization branches out, pages 74-81, 2004.
Cited by
Sorted by citations
MedReadCtrl: Personalizing medical text generation with readability-controlled instruction learningPreprint
Tran, H., Yao, Z., Jang, W. S., Sultana, S., Chang, A., Zhang, Y., & Hong, Y. (2025) · medRxiv · doi:10.1101/2025.07.09.25331239 · Cited by 9 · see citing works
“Moreover, limitations such as evaluation bias, domain contamination, and interpretability issues have been documented in prior work 35–37 .”
Multimodal Sarcasm Dataset Generation for a Low-Resource Language: Swahili
Kamau, E. K., & Abbas, N. (2025) · Lecture notes in computer science · doi:10.1007/978-3-032-11442-6_17
Quantifying and Mitigating Self-Preference Bias of LLM Judges
Yang, J., Hu, Z., Qiu, C., Deng, Z., Jiao, X., & Zhou, T. (2026) · arXiv · doi:10.48550/arxiv.2604.22891
“Furthermore, biases can manifest as “pref-erence leakage” towards specific model families [32] or be exacerbated by inconsistent reporting protocols [33].”
AgentRoom: Concurrent Multi-Agent Coding in a CRDT-Backed Shared Workspace
Cho, S., & Lee, D. (2026) · arXiv · doi:10.48550/arxiv.2608.23740
YESciEval: Robust LLM-as-a-Judge for Scientific Question Answering
D'Souza, J., Giglou, H. B., & Münch, Q. (2025) · arXiv · doi:10.48550/arxiv.2505.14279
“Notably, no LLM eval exhibited bias toward its own generated synthesis (Li et al., 2025).”
CARE: Confounder-Aware Aggregation for Reliable LLM Evaluation
Zhao, J., Shin, C., Huang, T.-H., GNVV, S. S. S. N., & Sala, F. (2026) · arXiv · doi:10.48550/arxiv.2603.00039
- Authors
- Li, D., Sun, R., Huang, Y., Zhong, M., Jiang, B., Han, J., Zhang, X., Wang, W., & Liu, H.
- Affiliations
- Arizona State University (US)
- University of California, Los Angeles (US)
- University of Notre Dame (US)
- University of Illinois Urbana-Champaign (US)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2025
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2502.01534
- OpenAlex
- W4407176798
- Semantic Scholar
- 24e8029197b5f99126f0ef4e577d55a275405b36
- Cited by
- 2
- References
- 98
- Citations by year
- 2025: 2
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Social Sciences › Safety Research › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Keywords
- LLM-as-a-judge, synthetic data generation, contamination, LLM-based evaluation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-07.
Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-07.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review