The DOI matches its Crossref record; 6 of the 6 references it cites were found in a registry or an index.
TrueSkill™: A Bayesian Skill Rating System
Abstract
We present a new Bayesian skill rating system which can be viewed as a generalisation of the Elo system used in Chess. The new system tracks the uncertainty about player skills, explicitly models draws, can deal with any number of competing entities and can infer individual skills from team results. Inference is performed by approximate message passing on a factor graph representation of the model. We present experimental evidence on the increased accuracy and convergence speed of the system compared to Elo and report on our experience with the new rating system running in a large-scale commercial online gaming service under the name of TrueSkill.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Factor graphs and the sum-product algorithm
Kschischang, F. R., Frey, B. J., & Loeliger, H.-A. (2001) · IEEE Transactions on Information Theory · doi:10.1109/18.910572 · Cited by 6,486
Fedorov, V. V., Studden, W. J., & Klimko, E. M. (1972) · Cited by 2,092
The Method of Paired Comparisons.
Curnow, R. N., & David, H. A. (1964) · Journal of the Royal Statistical Society. Series A (General) · doi:10.2307/2344014 · Cited by 1,271
A family of algorithms for approximate bayesian inference
Minka, T. P., & Picard, R. W. (2001) · DSpace@MIT (Massachusetts Institute of Technology) · Cited by 882
Self-citationParameter Estimation in Large Dynamic Paired Comparison Experiments
Glickman, M. E. (1999) · Journal of the Royal Statistical Society Series C: Applied Statistics · doi:10.1111/1467-9876.00159 · Cited by 351
Constraint Classification: A New Approach to Multiclass Classification
Har-Peled, S., Roth, D., & Zimak, D. (2002) · Lecture Notes in Computer Science · doi:10.1007/3-540-36169-3_29 · Cited by 173
Cited by
Generative Agents: Interactive Simulacra of Human Behavior
Park, J. S., O'Brien, J., Cai, C. J., Morris, M. R., Liang, P., & Bernstein, M. S. (2023) · Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology · doi:10.1145/3586183.3606763 · Cited by 2,187
Brochu, E., Cora, V. M., & de Freitas, N. (2010) · arXiv (Cornell University) · doi:10.48550/arxiv.1012.2599 · Cited by 2,011
Measuring human perceptions of a large-scale urban region using machine learning
Zhang, F., Zhou, B., Liu, L., Liu, Y., Fung, H. H., Lin, H., & Ratti, C. (2018) · Landscape and Urban Planning · doi:10.1016/j.landurbplan.2018.08.020 · Cited by 810
A survey and critique of multiagent deep reinforcement learning
Hernandez-Leal, P., Kartal, B., & Taylor, M. E. (2019) · Autonomous Agents and Multi-Agent Systems · doi:10.1007/s10458-019-09421-1 · Cited by 671
Mathematics for Machine Learning
Deisenroth, M. P., Faisal, A. A., & Ong, C. S. (2020) · Cambridge University Press eBooks · doi:10.1017/9781108679930 · Cited by 573
Findings of the 2014 Workshop on Statistical Machine Translation
Bojar, O., Buck, C., Federmann, C., Haddow, B., Koehn, P., Leveling, J., Monz, C., Pecina, P., Post, M., Saint-Amand, H., et al. (2014) · Proceedings of the Ninth Workshop on Statistical Machine Translation · doi:10.3115/v1/w14-3302 · Cited by 529
Deep Learning the City: Quantifying Urban Perception at a Global Scale
Dubey, A., Naik, N., Parikh, D., Raskar, R., & Hidalgo, C. A. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46448-0_12 · Cited by 513
Streetscore -- Predicting the Perceived Safety of One Million Streetscapes
Naik, N., Philipoom, J., Raskar, R., & Hidalgo, C. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition Workshops · doi:10.1109/cvprw.2014.121 · Cited by 453
(2010) · Cited by 444
Emergent Tool Use From Multi-Agent Autocurricula
Baker, B., Kanitscheider, I., Markov, T., Wu, Y., Powell, G., McGrew, B., & Mordatch, I. (2019) · arXiv · doi:10.48550/arxiv.1909.07528 · Cited by 424
“Metrics like ELO (Elo, 1978) or Trueskill (Herbrich et al., 2007) can more reliably measure whether performance is improving relative to previous policy versions or other policies in a population; however, these metrics still do not give insight into whether improved performance stems from new adaptations or improving…”
BackgroundFindings of the 2017 Conference on Machine Translation (WMT17)
Bojar, O., Chatterjee, R., Federmann, C., Graham, Y., Haddow, B., Huang, S., Huck, M., Koehn, P., Liu, Q., Logacheva, V., et al. (2017) · Proceedings of the Second Conference on Machine Translation · doi:10.18653/v1/w17-4717 · Cited by 395
Gordon, A. D., Henzinger, T. A., Nori, A. V., & Rajamani, S. K. (2014) · Future of Software Engineering Proceedings · doi:10.1145/2593882.2593900 · Cited by 357
COMPUTING “ELO RATINGS” OF MOVE PATTERNS IN THE GAME OF GO1
Coulom, R. (2007) · ICGA Journal · doi:10.3233/icg-2007-30403 · Cited by 298
Pairwise ranking aggregation in a crowdsourced setting
Chen, X., Bennett, P. N., Collins-Thompson, K., & Horvitz, E. (2013) · Proceedings of the sixth ACM international conference on Web search and data mining · doi:10.1145/2433396.2433420 · Cited by 290
Crowdsourced Data Management: A Survey
Li, G., Wang, J., Zheng, Y., & Franklin, M. J. (2016) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2016.2535242 · Cited by 283
Multimodal Language Analysis with Recurrent Multistage Fusion
Liang, P. P., Liu, Z., Bagher Zadeh, A., & Morency, L.-P. (2018) · Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d18-1014 · Cited by 230
Best practices for the human evaluation of automatically generated text
van der Lee, C., Gatt, A., van Miltenburg, E., Wubben, S., & Krahmer, E. (2019) · Proceedings of the 12th International Conference on Natural Language Generation · doi:10.18653/v1/w19-8643 · Cited by 205
Artificial intelligence moving serious gaming: Presenting reusable game AI components
Westera, W., Prada, R., Mascarenhas, S., Santos, P. A., Dias, J., Guimarães, M., Georgiadis, K., Nyamsuren, E., Bahreini, K., Yumak, Z., et al. (2020) · Education and Information Technologies · doi:10.1007/s10639-019-09968-2 · Cited by 136
Evaluation Examples are not Equally Informative: How should that change NLP Leaderboards?
Rodriguez, P., Barrow, J., Hoyle, A. M., Lalor, J. P., Jia, R., & Boyd-Graber, J. (2021) · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.18653/v1/2021.acl-long.346 · Cited by 47
“The venerable ELO model (Glickman and Jones, 1999) and modern extensions (Herbrich et al., 2007) predict whether a player (subject) will defeat an opponent (item) with, again, a similar mathematical model.”
BackgroundPersonalisation methods in e-learning-A literature review
Mikić, V., Ilić, M., Kopanja, L., & Vesin, B. (2022) · Computer Applications in Engineering Education · doi:10.1002/cae.22566 · Cited by 39
“Some of the most frequently used models for student ranking and assessment are: Item response theory [43, 72, 141, 212, 261], Elo rating algorithm [216, 273, 289] and TrueSkill [124, 148, 193].”
MethodEvolving maps for match balancing in first person shooters
Lanzi, P. L., Loiacono, D., & Stucchi, R. (2014) · 2014 IEEE Conference on Computational Intelligence and Games · doi:10.1109/cig.2014.6932901 · Cited by 28
Robust Conditional GAN from Uncertainty-Aware Pairwise Comparisons
Han, L., Gao, R., Kim, M., Tao, X., Liu, B., & Metaxas, D. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i07.6723 · Cited by 13
Debating with More Persuasive LLMs Leads to More Truthful Answers
Khan, A., Hughes, J., Valentine, D., Ruis, L., Sachan, K., Radhakrishnan, A., Grefenstette, E., Bowman, S. R., Rocktäschel, T., & Perez, E. (2024) · arXiv · doi:10.48550/arxiv.2402.06782 · Cited by 13
Scaling Scaling Laws with Board Games
Jones, A. L. (2021) · arXiv · doi:10.48550/arxiv.2104.03113 · Cited by 7
Metadata
- Authors
- Herbrich, R., Minka, T., & Graepel, T.
- Venue
- Advances in Neural Information Processing Systems 19
- Publisher
- The MIT Press
- Published
- 2007-09-07
- Pages
- 569-576
- OpenAlex
- W2153975459
- Cited by
- 639
- References
- 6
- FWCI
- 17.42
- Type
- book chapter
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.