The DOI matches its Crossref record; 17 of the 17 references it cites were found in a registry or an index.
Superhuman AI for heads-up no-limit poker: Libratus beats top professionals
Abstract
Libratus versus humans: Pitting artificial intelligence (AI) against top human players demonstrates just how far AI has come. Brown and Sandholm built a poker-playing AI called Libratus that decisively beat four leading human professionals in the two-player variant of poker called heads-up no-limit Texas hold'em (HUNL). Over nearly 3 weeks, Libratus played 120,000 hands of HUNL against the human professionals, using a three-pronged approach that included precomputing an overall strategy, adapting the strategy to actual gameplay, and learning from its opponent. Science , this issue p. 418
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
A decision-theoretic generalization of on-line learning and an application to boosting
Freund, Y., & Schapire, R. E. (1997) · Journal of Computer and System Sciences · doi:10.1006/jcss.1997.1504 · Cited by 20,466
Mastering the game of Go with deep neural networks and tree search
Silver, D., Huang, A., Maddison, C. J., Guez, A., Sifre, L., van den Driessche, G., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M., et al. (2016) · Nature · doi:10.1038/nature16961 · Cited by 15,612
Equilibrium points in n -person games
Nash, J. F. (1950) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.36.1.48 · Cited by 7,560
Bandit Based Monte-Carlo Planning
Kocsis, L., & Szepesvári, C. (2006) · Lecture Notes in Computer Science · doi:10.1007/11871842_29 · Cited by 2,855
The Weighted Majority Algorithm
Littlestone, N., & Warmuth, M. K. (1994) · Information and Computation · doi:10.1006/inco.1994.1009 · Cited by 1,975
Contributions to the theory of games.
(1960) · Journal of the Franklin Institute · doi:10.1016/0016-0032(60)90366-5 · Cited by 1,824
Efficient Selectivity and Backup Operators in Monte-Carlo Tree Search
Coulom, R. (2007) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-75538-8_7 · Cited by 1,240
Campbell, M., Hoane, A. J., & Hsu, F.-H. (2002) · Artificial Intelligence · doi:10.1016/s0004-3702(01)00129-1 · Cited by 1,226
A Simple Adaptive Procedure Leading to Correlated Equilibrium
Hart, S., & Mas-Colell, A. (2000) · Econometrica · doi:10.1111/1468-0262.00153 · Cited by 1,088
An analysis of alpha-beta pruning
Knuth, D. E., & Moore, R. W. (1975) · Artificial Intelligence · doi:10.1016/0004-3702(75)90019-3 · Cited by 1,004
Research works printed but not linked: 34
- J. Nash “Non-cooperative games ” thesis Princeton University (1950).
- J. Shi M. Littman in CG ’00: Revised Papers from the Second International Conference on Computers and Games (Springer 2002) pp. 333–345.
- D. Billings et al . in Proceedings of the 18th International Joint Conference on Artificial Intelligence (IJCAI) (Morgan Kaufmann Publishers San Francisco 2003) pp. 661–668.
- A. Gilpin T. Sandholm in Proceedings of the National Conference on Artificial Intelligence (AAAI) (AAAI Press 2005) pp. 1684–1685.
- N. Burch M. Johanson M. Bowling in AAAI Conference on Artificial Intelligence (AAAI) (AAAI Press 2014) pp. 602–608.
- M. Moravcik M. Schmid K. Ha M. Hladik S. Gaukrodger in AAAI Conference on Artificial Intelligence (AAAI) (AAAI Press 2016).
- E. Jackson in AAAI Workshop on Computer Poker and Imperfect Information (AAAI Press 2014).
- M. Zinkevich M. Johanson M. H. Bowling C. Piccione in Proceedings of the Annual Conference on Neural Information Processing Systems (NIPS) (Neural Information Processing Systems Foundation Inc. 2007) pp. 1729–1736.
- 10.1137/S1052623403422285 · doi:10.1137/s1052623403422285
- 10.1287/moor.1100.0452 · doi:10.1287/moor.1100.0452
- 10.1007/s10107-010-0430-2 · doi:10.1007/s10107-010-0430-2
- C. Kroer K. Waugh F. Klnç-Karzan T. Sandholm in Proceedings of the ACM Conference on Economics and Computation (EC) (ACM New York 2017).
- O. Tammelin N. Burch M. Johanson M. Bowling in Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI) (AAAI Press 2015) pp. 645–652.
- A. Gilpin T. Sandholm T. B. Sørensen in Proceedings of the Seventh International Joint Conference on Autonomous Agents and Multiagent Systems (International Foundation for Autonomous Agents and Multiagent Systems 2008) vol. 2 pp. 911–918.
- D. Schnizlein M. Bowling D. Szafron in Proceedings of the Twenty-First International Joint Conference on Artificial Intelligence (AAAI Press 2009) pp. 278–284.
- S. Ganzfried T. Sandholm in Proceedings of the Twenty-Third International Joint Conference on Artificial Intelligence (AAAI Press 2013) pp. 120–128.
- N. Brown T. Sandholm in Proceedings of the Twenty-Eighth AAAI Conference on Artificial Intelligence (AAAI) (AAAI Press 2014) pp. 594–601.
- N. Brown T. Sandholm in Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence (IJCAI-16) (AAAI Press 2016) pp. 4238–4239.
- N. Brown S. Ganzfried T. Sandholm in Proceedings of the 2015 International Conference on Autonomous Agents and Multiagent Systems (International Foundation for Autonomous Agents and Multiagent Systems 2015) pp. 7–15.
- N. Brown S. Ganzfried T. Sandholm in AAAI Conference on Artificial Intelligence (AAAI) (AAAI Press 2015) pp. 4270–4271.
- M. Johanson N. Burch R. Valenzano M. Bowling in Proceedings of the 2013 International Conference on Autonomous Agents and Multiagent Systems (International Foundation for Autonomous Agents and Multiagent Systems 2013) pp. 271–278.
- M. Lanctot K. Waugh M. Zinkevich M. Bowling in Proceedings of the Annual Conference on Neural Information Processing Systems (NIPS) (Neural Information Processing Systems Foundation Inc. 2009) pp. 1078–1086.
- R. Gibson M. Lanctot N. Burch D. Szafron M. Bowling in Proceedings of the Twenty-Sixth AAAI Conference on Artificial Intelligence (AAAI Press 2012) pp. 1355–1361.
- M. Johanson N. Bard M. Lanctot R. Gibson M. Bowling in Proceedings of the 11th International Conference on Autonomous Agents and Multiagent Systems (International Foundation for Autonomous Agents and Multiagent Systems 2012) vol. 2 pp. 837–846.
- K. Waugh et al . in Symposium on Abstraction Reformulation and Approximation (SARA) (AAAI Press 2009).
- M. Johanson N. Bard N. Burch M. Bowling in Proceedings of the Twenty-Sixth AAAI Conference on Artificial Intelligence (AAAI Press 2012) pp. 1371–1379.
- S. Ganzfried T. Sandholm in International Conference on Autonomous Agents and Multiagent Systems (AAMAS) (International Foundation for Autonomous Agents and Multiagent Systems 2015) pp. 37–45.
- N. Brown, T. Sandholm, Adv. Neural Inf. Process. Syst. 30, 689–699 (2017).
- D. Billings D. Papp J. Schaeffer D. Szafron in Proceedings of the National Conference on Artificial Intelligence (AAAI) (AAAI Press 1998) pp. 493–499.
- S. Ganzfried T. Sandholm in International Conference on Autonomous Agents and Multiagent Systems (AAMAS) (International Foundation for Autonomous Agents and Multiagent Systems 2011).
- M. Johanson K. Waugh M. Bowling M. Zinkevich in Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI) (AAAI Press 2011) pp. 258–265.
- N. Brown T. Sandholm in Proceedings of the Annual Conference on Neural Information Processing Systems (NIPS) (2015) pp. 1972–1980.
- N. Brown T. Sandholm in International Conference on Machine Learning (Proceedings of Machine Learning Research 2017).
- S. Ganzfried T. Sandholm K. Waugh in International Conference on Autonomous Agents and Multiagent Systems (AAMAS) (International Foundation for Autonomous Agents and Multiagent Systems 2012) pp. 871–878.
Cited by
Multi-Agent Reinforcement Learning: A Selective Overview of Theories and Algorithms
Zhang, K., Yang, Z., & Başar, T. (2021) · Studies in Systems, Decision and Control · doi:10.1007/978-3-030-60990-0_12 · Cited by 1,264
Mastering Atari, Go, chess and shogi by planning with a learned model
Schrittwieser, J., Antonoglou, I., Hubert, T., Simonyan, K., Sifre, L., Schmitt, S., Guez, A., Lockhart, E., Hassabis, D., Graepel, T., et al. (2020) · Nature · doi:10.1038/s41586-020-03051-4 · Cited by 1,214
A survey and critique of multiagent deep reinforcement learning
Hernandez-Leal, P., Kartal, B., & Taylor, M. E. (2019) · Autonomous Agents and Multi-Agent Systems · doi:10.1007/s10458-019-09421-1 · Cited by 671
Superhuman AI for multiplayer poker
Brown, N., & Sandholm, T. (2019) · Science · doi:10.1126/science.aay2400 · Cited by 592
Self-citationOpportunities and Challenges for Machine Learning in Materials Science
Morgan, D., & Jacobs, R. (2020) · Annual Review of Materials Research · doi:10.1146/annurev-matsci-070218-010015 · Cited by 549
Trust in AI: progress, challenges, and future directions
Afroogh, S., Akbari, A., Malone, E., Kargar, M., & Alambeigi, H. (2024) · Humanities and Social Sciences Communications · doi:10.1057/s41599-024-04044-8 · Cited by 507
Wu, S., Kondo, Y., Kakimoto, M.-A., Yang, B., Yamada, H., Kuwajima, I., Lambard, G., Hongo, K., Xu, Y., Shiomi, J., et al. (2019) · npj Computational Materials · doi:10.1038/s41524-019-0203-2 · Cited by 478
Finding key players in complex networks through deep reinforcement learning
Fan, C., Zeng, L., Sun, Y., & Liu, Y.-Y. (2020) · Nature Machine Intelligence · doi:10.1038/s42256-020-0177-2 · Cited by 425
Dimiduk, D. M., Holm, E. A., & Niezgoda, S. R. (2018) · Integrating Materials and Manufacturing Innovation · doi:10.1007/s40192-018-0117-8 · Cited by 347
Machine learning in materials science: From explainable predictions to autonomous design
Pilania, G. (2021) · Computational Materials Science · doi:10.1016/j.commatsci.2021.110360 · Cited by 281
Human-level play in the game of Diplomacy by combining language models with strategic reasoning
Meta Fundamental AI Research Diplomacy Team (FAIR)†, Bakhtin, A., Brown, N., Dinan, E., Farina, G., Flaherty, C., Fried, D., Goff, A., Gray, J., Hu, H., et al. (2022) · Science · doi:10.1126/science.ade9097 · Cited by 241
Self-citationHybrid collective intelligence in a human–AI society
Peeters, M. M. M., van Diggelen, J., van den Bosch, K., Bronkhorst, A., Neerincx, M. A., Schraagen, J. M., & Raaijmakers, S. (2021) · AI & SOCIETY · doi:10.1007/s00146-020-01005-y · Cited by 229
AI strategy of India: policy framework, adoption challenges and actions for government
Chatterjee, S. (2020) · Transforming Government: People, Process and Policy · doi:10.1108/tg-05-2019-0031 · Cited by 142
Impact of AI regulation on intention to use robots: From citizens and government perspective
Chatterjee, S. (2020) · International Journal of Intelligent Unmanned Systems · doi:10.1108/ijius-09-2019-0051 · Cited by 81
The First Minds: Caterpillars, Karyotes, and Consciousness
Reber, A. S. (2018) · doi:10.1093/oso/9780190854157.001.0001 · Cited by 65
The AI of the Beholder: Intra‐Professional Sensemaking of an Epistemic Technology
Scarbrough, H., Chen, Y., & Patriotta, G. (2025) · Journal of Management Studies · doi:10.1111/joms.13065 · Cited by 53
Effects of Communication Directionality and AI Agent Differences in Human-AI Interaction
Ashktorab, Z., Dugan, C., Johnson, J., Pan, Q., Zhang, W., Kumaravel, S., & Campbell, M. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445256 · Cited by 50
Finding Friend and Foe in Multi-Agent Games
Serrino, J., Kleiman-Weiner, M., Parkes, D. C., & Tenenbaum, J. B. (2019) · arXiv · doi:10.48550/arxiv.1906.02330 · Cited by 33
MAESTRO: Open-Ended Environment Design for Multi-Agent Reinforcement Learning
Samvelyan, M., Khan, A., Dennis, M., Jiang, M., Parker-Holder, J., Foerster, J., Raileanu, R., & Rocktäschel, T. (2023) · arXiv · doi:10.48550/arxiv.2303.03376 · Cited by 8
“Zinkevich et al. (2007) use self-play with regret minimisation for achieving Nash equilibrium, an approach that led to superhuman performance in Poker (Brown & Sandholm, 2018; 2019).”
MethodGreen AI: Balancing Model Complexity and Energy Footprint in Deep Learning
Aggarwal, D., Sharma, D., & Saxena, A. B. (2025) · 2025 3rd International Conference on Sustainable Computing and Data Communication Systems (ICSCDS) · doi:10.1109/icscds65426.2025.11167425
Metadata
- Authors
- Brown, N., & Sandholm, T.
- Venue
- Science
- Published
- 2018-01-26
- Volume
- 359
- Issue
- 6374
- Pages
- 418-424
- ISSN
- 0036-8075 (ISSN-L)
- 1095-9203
- OpenAlex
- W2773381986
- Cited by
- 540
- References
- 17
- FWCI
- 31.16
- Type
- journal article
- Language
- en
- Open access
- closed
- Funders
- National Science Foundation (IIS-1617590, IIS-1718457, CCF-1733556)
- Army Research Office (W911NF-17-1-0082)
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.