Parameter Estimation in Large Dynamic Paired Comparison Experiments
Open accessThe DOI matches its Crossref record; 15 of the 15 references it cites were found in a registry or an index.
Abstract
SUMMARY: Paired comparison data in which the abilities or merits of the objects being compared may be changing over time can be modelled as a non-linear state space model. When the population of objects being compared is large, likelihood-based analyses can be too computationally cumbersome to carry out regularly. This presents a problem for rating populations of chess players and other large groups which often consist of tens of thousands of competitors. This problem is overcome through a computationally simple non-iterative algorithm for fitting a particular dynamic paired comparison model. The algorithm, which improves over the commonly used algorithm of Elo by incorporating the variability in parameter estimates, can be performed regularly even for large populations of competitors. The method is evaluated on simulated data and is applied to ranking the best chess players of all time, and to ranking the top current tennis-players.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
A Simplex Method for Function Minimization
Nelder, J. A., & Mead, R. (1965) · The Computer Journal · doi:10.1093/comjnl/7.4.308 · Cited by 28,905
Rank Analysis of Incomplete Block Designs: I. The Method of Paired Comparisons
Bradley, R. A., & Terry, M. E. (1952) · Biometrika · doi:10.2307/2334029 · Cited by 2,412
RANK ANALYSIS OF INCOMPLETE BLOCK DESIGNS: THE METHOD OF PAIRED COMPARISONS
BRADLEY, R. A., & TERRY, M. E. (1952) · Biometrika · doi:10.1093/biomet/39.3-4.324 · Cited by 1,434
The Method of Paired Comparisons.
Curnow, R. N., & David, H. A. (1964) · Journal of the Royal Statistical Society. Series A (General) · doi:10.2307/2344014 · Cited by 1,271
(1978) · Medical Entomology and Zoology · Cited by 1,199
Bayesian Statistics without Tears: A Sampling–Resampling Perspective
Smith, A. F. M., & Gelfand, A. E. (1992) · The American Statistician · doi:10.1080/00031305.1992.10475856 · Cited by 1,003
The Method of Paired Comparisons
Christensen, R. (1989) · Technometrics · doi:10.1080/00401706.1989.10488612 · Cited by 450
On Extending the Bradley-Terry Model to Accommodate Ties in Paired Comparison Experiments
Davidson, R. R. (1970) · Journal of the American Statistical Association · doi:10.1080/01621459.1970.10481082 · Cited by 361
Ties in Paired-Comparison Experiments: A Generalization of the Bradley-Terry Model
Rao, P. V., & Kupper, L. L. (1967) · Journal of the American Statistical Association · doi:10.1080/01621459.1967.10482901 · Cited by 352
14 Paired comparisons: Some basic procedures and examples
BRADLEY, R. A. (1984) · Handbook of statistics · doi:10.1016/s0169-7161(84)04016-5 · Cited by 129
Cited by
Review on Computational Trust and Reputation Models
Sabater, J., & Sierra, C. (2005) · Artificial Intelligence Review · doi:10.1007/s10462-004-0041-5 · Cited by 911
TrueSkill™: A Bayesian Skill Rating System
Herbrich, R., Minka, T., & Graepel, T. (2007) · Advances in Neural Information Processing Systems 19 · doi:10.7551/mitpress/7503.003.0076 · Cited by 639
Trust management through reputation mechanisms
Zacharia, G., & Maes, P. (2000) · Applied Artificial Intelligence · doi:10.1080/08839510050144868 · Cited by 521
Klinkenberg, S., Straatemeier, M., & van der Maas, H. L. J. (2011) · Computers & Education · doi:10.1016/j.compedu.2011.02.003 · Cited by 257
Empirical analysis of programming language adoption
Meyerovich, L. A., & Rabkin, A. S. (2013) · Proceedings of the 2013 ACM SIGPLAN international conference on Object oriented programming systems languages & applications · doi:10.1145/2509136.2509515 · Cited by 158
“” As described in Section 2, we use the Glicko-2 algorithm [8] to totally order all the languages for each such statement [14].”
MethodTemporal dynamics of social hierarchy formation and maintenance in male mice
Williamson, C. M., Lee, W., & Curley, J. P. (2016) · Animal Behaviour · doi:10.1016/j.anbehav.2016.03.004 · Cited by 150
So, N., Franks, B., Lim, S., & Curley, J. P. (2015) · PLOS ONE · doi:10.1371/journal.pone.0134509 · Cited by 135
Social context-dependent relationships between mouse dominance rank and plasma hormone levels
Williamson, C. M., Lee, W., Romeo, R. D., & Curley, J. P. (2017) · Physiology & Behavior · doi:10.1016/j.physbeh.2016.12.038 · Cited by 120
“Briefly, all individuals in each group start with the same initial rating and gain or lose points following each agonistic interaction based on the rating difference between themselves and the individual they defeat or lose to [24,40].”
BackgroundMethodInfluentialWilliamson, C. M., Lee, W., DeCasien, A. R., Lanham, A., Romeo, R. D., & Curley, J. P. (2019) · Scientific Reports · doi:10.1038/s41598-019-43747-w · Cited by 99
A Revealed Preference Ranking of U.S. Colleges and Universities *
Avery, C. N., Glickman, M. E., Hoxby, C. M., & Metrick, A. (2013) · The Quarterly Journal of Economics · doi:10.1093/qje/qjs043 · Cited by 97
Self-citationThe deterministic part of the seventh International Planning Competition
Linares López, C., Jiménez Celorrio, S., & García Olaya, Á. (2015) · Artificial Intelligence · doi:10.1016/j.artint.2015.01.004 · Cited by 71
“We tried to use a more informative measure that took into account which specific planners solved which problems based on the Glicko rating system [32].”
MethodWilliamson, C. M., Romeo, R. D., & Curley, J. P. (2017) · Hormones and Behavior · doi:10.1016/j.yhbeh.2016.11.001 · Cited by 62
Temporal pairwise-correlation analysis provides empirical support for attention hierarchies in mice
Curley, J. P. (2016) · Biology Letters · doi:10.1098/rsbl.2016.0192 · Cited by 43
The dynamics of dominance: open questions, challenges and solutions
Strauss, E. D., & Shizuka, D. (2022) · Philosophical Transactions of the Royal Society B · doi:10.1098/rstb.2020.0445 · Cited by 42
Williamson, C. M., Klein, I. S., Lee, W., & Curley, J. P. (2019) · Social Neuroscience · doi:10.1080/17470919.2018.1479303 · Cited by 37
A study of forecasting tennis matches via the Glicko model
Yue, J. C., Chou, E. P., Hsieh, M.-H., & Hsiao, L.-C. (2022) · PLOS ONE · doi:10.1371/journal.pone.0266838 · Cited by 22
“Another rating system is the Glicko model [16], developed by Mark Glickman to improve the Elo rating system.”
Metadata
- Authors
- Glickman, M. E.
- Venue
- Journal of the Royal Statistical Society Series C: Applied Statistics
- Publisher
- Oxford University Press (OUP)
- Published
- 1999-09-01
- Volume
- 48
- Issue
- 3
- Pages
- 377-394
- ISSN
- 0035-9254 (ISSN-L)
- 1467-9876
- OpenAlex
- W2162185655
- Cited by
- 351
- References
- 15
- FWCI
- 11.33
- Type
- journal article
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.