The DOI matches its Crossref record; 244 of the 244 references it cites were found in a registry or an index.
Bandit Algorithms
Abstract
Decision-making in the face of uncertainty is a significant challenge in machine learning, and the multi-armed bandit model is a commonly used framework to address it. This comprehensive and rigorous introduction to the multi-armed bandit problem examines all the major settings, including stochastic, adversarial, and Bayesian frameworks. A focus on both mathematical intuition and carefully worked proofs makes this an excellent reference for established researchers and a helpful resource for graduate students in computer science, engineering, statistics, applied mathematics and economics. Linear bandits receive special attention as one of the most useful models in applications, while other chapters are dedicated to combinatorial bandits, ranking, non-stationary problems, Thompson sampling and pure exploration. The book ends with a peek into the world beyond bandits with an introduction to partial monitoring and learning in Markov decision processes.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Boyd, S., & Vandenberghe, L. (2004) · doi:10.1017/cbo9780511804441 · Cited by 56,601
Prospect Theory: An Analysis of Decision under Risk
Kahneman, D., & Tversky, A. (1979) · Econometrica · doi:10.2307/1914185 · Cited by 48,524
Gelman, A., Carlin, J. B., Stern, H. S., & Rubin, D. B. (2003) · doi:10.1201/9780429258480 · Cited by 27,276
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225
Wu, Y. (1999) · Technometrics · doi:10.1080/00401706.1999.10485951 · Cited by 27,221
Mastering the game of Go with deep neural networks and tree search
Silver, D., Huang, A., Maddison, C. J., Guez, A., Sifre, L., van den Driessche, G., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M., et al. (2016) · Nature · doi:10.1038/nature16961 · Cited by 15,612
Advances in Neural Information Processing Systems 19
(2007) · The MIT Press eBooks · doi:10.52202/085713 · Cited by 15,539
Advances in neural information processing systems 7
(1996) · Computers & Mathematics with Applications · doi:10.1016/s0898-1221(96)90463-0 · Cited by 14,281
Advances in Neural Information Processing Systems 29
Teymur, O., Zygalakis, K., & Calderhead, B. (2016) · Cited by 13,887
Advances in Neural Information Processing Systems 25
Zhang, Y., Sutton, C. A., Storkey, A., & Zoubin Ghahramani, A. (2012) · Cited by 12,653
Cited by
Degenerate Feedback Loops in Recommender Systems
Jiang, R., Chiappa, S., Lattimore, T., György, A., & Kohli, P. (2019) · Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3306618.3314288 · Cited by 232
“We see that the Optimal Oracle results in the fastest degeneration by far, followed by the Oracle, TS and UCB .”
MethodInfluentialSelf-citationA Survey of Reinforcement Learning from Human Feedback
Kaufmann, T., Weng, P., Bengs, V., & Hüllermeier, E. (2023) · arXiv · doi:10.48550/arxiv.2312.14925 · Cited by 43
Kadi, H. A., & Terzić, K. (2023) · Sensors · doi:10.3390/s23052389 · Cited by 27
“Some fundamental myopic exploration strategies are derived from the Multi-armed Bandit (MAB) formulation [196], then scaled to more complex MDP/POMDP settings.”
MethodConformal Decision Theory: Safe Autonomous Decisions from Imperfect Predictions
Lekeufack, J., Angelopoulos, A. N., Bajcsy, A., Jordan, M. I., & Malik, J. (2024) · 2024 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra57147.2024.10610041 · Cited by 18
“We compare our method with two baselines: A bandit algorithm running the upper confidence bound algorithm ( UCB ) [46] to maximize the utility V and another algorithm running the lower confidence bound algorithm ( LCB ) to minimize the loss L .”
MethodSublinear regret for learning POMDPs
Xiong, Y., Chen, N., Gao, X., & Zhou, X. (2022) · Production and Operations Management · doi:10.1111/poms.13778 · Cited by 6
Continual Learning as Computationally Constrained Reinforcement Learning
Kumar, S., Marklund, H., Rao, A., Zhu, Y., Jeon, H. J., Liu, Y., & Van Roy, B. (2023) · arXiv · doi:10.48550/arxiv.2307.04345 · Cited by 4
Efficient Prompt Optimization Through the Lens of Best Arm Identification
Shi, C., Yang, K., Chen, Z., Li, J., Yang, J., & Shen, C. (2024) · Advances in Neural Information Processing Systems 37 · doi:10.52202/079017-3161 · Cited by 3
“The research of multi-armed bandits (MAB) has a long and rich history; see representative surveys of Lattimore and Szepesvári [41], Bubeck et al. [10].”
Influential
Metadata
- Authors
- Lattimore, T., & Szepesvári, C.
- Venue
- Cambridge University Press eBooks
- Publisher
- Cambridge University Press
- Published
- 2020-07-04
- OpenAlex
- W4206530644
- Cited by
- 976
- References
- 244
- FWCI
- 55.96
- Type
- book
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.