RANK ANALYSIS OF INCOMPLETE BLOCK DESIGNS: THE METHOD OF PAIRED COMPARISONS
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The Bradley-Terry coefficient, computed from a logistic regression model, is used to compare performance and uncertainty estimates of pairwise evaluations in LLMs and other settings.
“In particular, the Bradley-Terry coefficient (Bradley and Terry, 1952) - computed from a logistic regression model - is used to compare the performance and uncertainty estimates of pairwise evaluations in LLMs as well as other settings (Chiang et al., 2024; Frick et al., 2025).”
From FLAWS: A Benchmark for Error Identification and Localization in Scientific Papers (2025) · cites this paper as Bradley and Terry, 1952
Pairwise preference learning provides relative supervision that can complement direct scores.
“Pairwise preference learning provides relative supervision that can complement direct scores (Bradley and Terry, 1952;Jeong et al., 2025).”
From JudgeMoE: Distributional Aggregation for LLM-as-a-Judge (2026) · cites this paper as Bradley and Terry, 1952
Cited by
Sorted by citations
Huang, L., Yu, W., Ma, W., Zhong, W., Feng, Z., Wang, H., Chen, Q., Peng, W., Feng, X., Qin, B., et al. (2025) · ACM Transactions on Information Systems · doi:10.1145/3703155 · Cited by 2,142 · see citing works
Default Bayes factors for ANOVA designs
Rouder, J. N., Morey, R. D., Speckman, P. L., & Province, J. M. (2012) · Journal of Mathematical Psychology · doi:10.1016/j.jmp.2012.08.001 · Cited by 1,800 · see citing works
Deep reinforcement learning from human preferences
Christiano, P., Leike, J., Brown, T. B., Martic, M., Legg, S., & Amodei, D. (2017) · arXiv · doi:10.48550/arxiv.1706.03741 · Cited by 1,058 · see citing works
Jain, A., Hautier, G., Ong, S. P., & Persson, K. (2016) · Journal of Materials Research · doi:10.1557/jmr.2016.80 · Cited by 244 · see citing works
Huang, L., Yu, W., Ma, W., Zhong, W., Feng, Z., Wang, H., Chen, Q., Peng, W., Feng, X., Qin, B., et al. (2023) · arXiv · doi:10.48550/arxiv.2311.05232 · Cited by 2,516 · see citing works
The LambdaLoss Framework for Ranking Metric Optimization
Wang, X., Li, C., Golbandi, N., Bendersky, M., & Najork, M. (2018) · Proceedings of the 27th ACM International Conference on Information and Knowledge Management · doi:10.1145/3269206.3271784 · Cited by 136 · see citing works
RLHF Deciphered: A Critical Analysis of Reinforcement Learning from Human Feedback for LLMs
Chaudhari, S., Aggarwal, P., Murahari, V., Rajpurohit, T., Kalyan, A., Narasimhan, K., Deshpande, A., & Castro da Silva, B. (2026) · ACM Computing Surveys · doi:10.1145/3743127 · Cited by 92 · see citing works
“The challenges encountered in learning these reward functions highlight both the practical and fundamental limitations of RLHF, as thoroughly analyzed by Casper et al. [24].”
Corticosteroid-sparing therapy: practice patterns among uveitis specialists
Esterberg, E., & Acharya, N. R. (2012) · Journal of Ophthalmic Inflammation and Infection · doi:10.1007/s12348-011-0047-5 · Cited by 58 · see citing works
Lopez, M. J., Matthews, G. J., & Baumer, B. S. (2018) · The Annals of Applied Statistics · doi:10.1214/18-aoas1165 · Cited by 52 · see citing works
Hallucination of Multimodal Large Language Models: A Survey
Bai, Z., Wang, P., Xiao, T., He, T., Han, Z., Zhang, Z., & Shou, M. Z. (2024) · arXiv · doi:10.48550/arxiv.2404.18930 · Cited by 38 · see citing works
“RLHF typically employs a preference model [7], trained to predict preference rankings based on prompts and human-labeled responses.”
ReviewerGPT? An Exploratory Study on Using Large Language Models for Paper Reviewing
Liu, R., & Shah, N. B. (2023) · arXiv · doi:10.48550/arxiv.2306.00622 · Cited by 23 · see citing works
Fine-tuning Language Models for Factuality
Tian, K., Mitchell, E., Yao, H., Manning, C. D., & Finn, C. (2023) · arXiv · doi:10.48550/arxiv.2311.08401 · Cited by 14 · see citing works
“The probability of observing a particular preference pair is assumed to follow a Bradley-Terry model (Bradley & Terry, 1952): where σ is the sigmoid function and r ( x, y ) is an unobserved reward or scoring function.”
RouteLLM: Learning to Route LLMs with Preference Data
Ong, I., Almahairi, A., Wu, V., Chiang, W.-L., Wu, T., Gonzalez, J. E., Kadous, M. W., & Stoica, I. (2024) · arXiv · doi:10.48550/arxiv.2406.18665 · Cited by 15 · see citing works
“We adopt a Bradley-Terry (BT) model [6] similar to [10].”
Mitigating Sycophancy in Large Language Models via Direct Preference Optimization
Khan, A. A., Alam, S., Wang, X., Khan, A. F., Neog, D. R., & Anwar, A. (2024) · 2024 IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata62323.2024.10825538 · Cited by 6 · see citing works
Han, B., Ma, Y., Coghlan, S., McKay, D., Buchanan, G., & Smith, W. (2026) · Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3790360 · Cited by 3 · see citing works
Search Arena: Analyzing Search-Augmented LLMs
Miroyan, M., Wu, T.-H., King, L., Li, T., Pan, J., Hu, X., Chiang, W.-L., Angelopoulos, A. N., Darrell, T., Norouzi, N., et al. (2025) · arXiv · doi:10.48550/arxiv.2506.05334 · Cited by 1 · see citing works
“Your response: Consistent with Chatbot Arena’s ranking system [10], we use Bradley-Terry regression [5] to calculate model coefficients and then re-scale to match the scale of the Elo rating system.”
RewardBench 2: Advancing Reward Model Evaluation
Malik, S., Pyatkin, V., Land, S., Morrison, J., Smith, N. A., Hajishirzi, H., & Lambert, N. (2025) · arXiv · doi:10.48550/arxiv.2506.01937 · Cited by 1 · see citing works
“With that data, a reward model r ∗ is trained to output a scalar value to predict the probability p ∗ of a prompt and completion falling in the chosen category, following a Bradley-Terry model of human preferences [17]: .”
FLAWS: A Benchmark for Error Identification and Localization in Scientific Papers
Xi, S., Rao, V., Payan, J., & Shah, N. B. (2025) · arXiv · doi:10.48550/arxiv.2511.21843
“In particular, the Bradley-Terry coefficient (Bradley and Terry, 1952) — computed from a logistic regression model — is used to compare the performance and uncertainty estimates of pairwise evaluations in LLMs as well as other settings (Chiang et al., 2024 et al., 2025).”
Kawamura, M., Shirahata, Y., Mitsui, K., & Shimizu, R. (2026) · arXiv · doi:10.48550/arxiv.2606.20137
“We train the accent-quality head using a pairwise logistic ranking loss, inspired by the Bradley–Terry model [19].”
MAPLE: Multi-Aspect Panels of LLM Evaluators for Open-Ended Questions
Jinji, M., Atarashi, K., Takeuchi, K., & Kashima, H. (2026) · Findings of the Association for Computational Linguistics: ACL 2026 · doi:10.18653/v1/2026.findings-acl.1351
“These comparisons are then aggregated using a reliability-aware statistical model that extends Crowd-BT (Chen et al., 2013) based on the Bradley-Terry model (Bradley and Terry, 1952).”
JudgeMoE: Distributional Aggregation for LLM-as-a-Judge
Liu, Y., James, J., Wang, Y., Zhao, K., Xiao, C., & Lin, C. (2026) · arXiv · doi:10.48550/arxiv.2610.07109
“Pairwise preference learning provides relative supervision that can complement direct scores (Bradley and Terry, 1952; Jeong et al., 2025).”
Base Models Beat Aligned Models at Randomness and Creativity
West, P., & Potts, C. (2025) · arXiv · doi:10.48550/arxiv.2505.00047
T2I-Eval-R1: Reinforcement Learning-Driven Reasoning for Interpretable Text-to-Image Evaluation
Ma, Z.-A., Lan, T., Tu, R.-C., Liu, S.-H., Huang, H., Wu, Z., Xu, C., & Mao, X.-L. (2025) · arXiv · doi:10.48550/arxiv.2505.17897
Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context
Gemini Team, Georgiev, P., Lei, V. I., Burnell, R., Bai, L., Gulati, A., Tanzer, G., Vincent, D., Pan, Z., Wang, S., et al. (2024) · arXiv · doi:10.48550/arxiv.2403.05530 · Cited by 306 · see citing works
- Authors
- BRADLEY, R. A., & TERRY, M. E.
- Affiliations
- Virginia Agricultural Experiment Station (US)
- Venue
- Biometrika
- Publisher
- Oxford University Press (OUP)
- Published
- 1952
- Volume
- 39
- Issue
- 3-4
- Pages
- 324-345
- ISSN
- 0006-3444 (ISSN-L)
- 1464-3510
- OpenAlex
- W2013784666
- Cited by
- 1,434
- References
- 0
- Citations by year
- 2026: 60 · 2025: 55 · 2024: 37 · 2023: 40 · 2022: 49 · 2021: 77 · 2020: 90 · 2019: 56 · 2018: 65 · 2017: 75 · 2016: 79 · 2015: 71 · 2014: 38 · 2013: 50 · 2012: 40
- FWCI
- 0.65
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Biometrika (publishedVersion)
- Topics
- Social Sciences › Decision Sciences › Management Science and Operations Research › Optimal Experimental Design Methods
- Physical Sciences › Mathematics › Statistics and Probability › Probability and Statistical Research
- Physical Sciences › Mathematics › Statistics and Probability › Advanced Statistical Methods and Models
- Keywords
- rank analysis, incomplete block designs, rank deficiency, block designs, linear models
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
