The DOI matches its Crossref record; 244 of the 244 references it cites were found in a registry or an index.
Bandit Algorithms
Abstract
Decision-making in the face of uncertainty is a significant challenge in machine learning, and the multi-armed bandit model is a commonly used framework to address it. This comprehensive and rigorous introduction to the multi-armed bandit problem examines all the major settings, including stochastic, adversarial, and Bayesian frameworks. A focus on both mathematical intuition and carefully worked proofs makes this an excellent reference for established researchers and a helpful resource for graduate students in computer science, engineering, statistics, applied mathematics and economics. Linear bandits receive special attention as one of the most useful models in applications, while other chapters are dedicated to combinatorial bandits, ranking, non-stationary problems, Thompson sampling and pure exploration. The book ends with a peek into the world beyond bandits with an introduction to partial monitoring and learning in Markov decision processes.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Items 201–244 of 244
Path Kernels and Multiplicative Updates
Takimoto, E., & Warmuth, M. K. (2002) · Lecture notes in computer science · doi:10.1007/3-540-45435-7_6 · Cited by 103
An asymptotically optimal policy for finite support models in the multiarmed bandit problem
Honda, J., & Akimichi Takemura, Y. (2011) · Machine Learning · doi:10.1007/s10994-011-5257-4 · Cited by 102
Lipschitz Bandits: Regret Lower Bound and Optimal Algorithms
Magureanu, S., Combes, R., & Proutière, A. (2014) · Conference on Learning Theory · Cited by 99
Competitive-Cooperative-Concurrent Reinforcement Learning with Importance Sampling
Uchibe, E., & Doya, K. (2004) · The MIT Press eBooks · doi:10.7551/mitpress/3122.003.0037 · Cited by 99
Information Directed Sampling and Bandits with Heteroscedastic NoisePreprint
Kirschner, J., & Krause, A. (2018) · arXiv (Cornell University) · doi:10.3929/ethz-b-000310984 · Cited by 99
A generalized Gittins index for a Markov chain and its recursive calculation
Sonin, I. M. (2008) · Statistics & Probability Letters · doi:10.1016/j.spl.2008.01.049 · Cited by 96
Online Learning for Linearly Parametrized Control Problems
Abbasi-Yadkori, Y. (2013) · ERA: Education and Research Archive (University of Alberta) · doi:10.7939/r3h418 · Cited by 96
Bernoulli Rank-1 Bandits for Click Feedback
Katariya, S., Kveton, B., Szepesvári, C., Vernade, C., & Wen, Z. (2017) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2017/278 · Cited by 96
Self-citationHelmbold, D. P., Littlestone, N., & Long, P. M. (2000) · Information and Computation · doi:10.1006/inco.2000.2870 · Cited by 94
Festschrift for Lucien Le Cam: Research Papers in Probability and Statistics.
Kafadar, K., Polland, D., Torgersen, E. N., et al. (1997) · Journal of the American Statistical Association · doi:10.2307/2965446 · Cited by 91
Interior-Point Methods for Full-Information and Bandit Online Learning
Abernethy, J., Hazan, E., & Rakhlin, A. (2012) · IEEE Transactions on Information Theory · doi:10.1109/tit.2012.2192096 · Cited by 88
On the Complexity of Bandit and Derivative-Free Stochastic Convex OptimizationPreprint
Shamir, O. (2012) · arXiv (Cornell University) · doi:10.48550/arxiv.1209.2388 · Cited by 86
Computing a Classic Index for Finite-Horizon Bandits
Niño‐Mora, J. (2010) · INFORMS journal on computing · doi:10.1287/ijoc.1100.0398 · Cited by 84
Multi‐Armed Bandits, Gittins Index, and its Calculation
Chakravorty, J., & Mahajan, A. (2014) · Wiley eBooks · doi:10.1002/9781118596333.ch24 · Cited by 82
Sequential Learning for Multi-Channel Wireless Network Monitoring With Channel Switching Costs
Le, T. V., Szepesvári, C., & Zheng, R. (2014) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2014.2357779 · Cited by 82
Self-citationBounded Regret for Finite-Armed Structured BanditsPreprint
Lattimore, T., & Munos, R. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1411.2919 · Cited by 82
Self-citationThompson Sampling for Stochastic Control: The Finite Parameter Case
Kim, M. J. (2017) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2017.2653942 · Cited by 82
Informational confidence bounds for self-normalized averages and applications
Garivier, A. (2013) · IEEE Information Theory Workshop (ITW) · doi:10.1109/itw.2013.6691311 · Cited by 79
An asymptotically optimal algorithm for the max k-armed bandit problem
Streeter, M. J. V., & Smith, S. F. (2006) · Cited by 78
Risk-Sensitive and Risk-Neutral Multiarmed Bandits
Denardo, E. V., Park, H., & Rothblum, U. G. (2007) · Mathematics of Operations Research · doi:10.1287/moor.1060.0240 · Cited by 78
Bandit Algorithms in Information Retrieval
Głowacka, D. (2019) · now publishers, Inc. eBooks · doi:10.1561/9781680835755 · Cited by 78
Nearly Minimax-Optimal Regret for Linearly Parameterized Bandits
Li, Y., Wang, Y., & Zhou, Y. (2023) · IEEE Transactions on Information Theory · doi:10.1109/tit.2023.3267732 · Cited by 78
Linear Programming for Finite State Multi-Armed Bandit Problems
Chen, Y. R., & Katehakis, M. N. (1986) · Mathematics of Operations Research · doi:10.1287/moor.11.1.180 · Cited by 76
Artificial Intelligence in Education
Guo, Y., & Xiao, Y. (2019) · International Conference on Modern Management, Education Technology and Social Science (MMETSS) · doi:10.2991/mmetss-19.2019.33 · Cited by 73
Finite-time lower bounds for the two-armed bandit problem
Kulkarni, S. R., & Lugosi, G. (2000) · IEEE Transactions on Automatic Control · doi:10.1109/9.847107 · Cited by 72
Helmbold, D. P., & Williamson, B. (2001) · Lecture notes in computer science · doi:10.1007/3-540-44581-1 · Cited by 71
Regret Analysis of the Finite-Horizon Gittins Index Strategy for Multi-Armed BanditsPreprint
Lattimore, T. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1511.06014 · Cited by 71
Self-citationTwo-Target Algorithms for Infinite-Armed Bandits with Bernoulli Rewards
Bonald, T., & Paristech, T. (2013) · Cited by 69
Kocák, T., Valko, M., Munos, R., & Agrawal, S. (2014) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v28i1.9011 · Cited by 69
ASYMPTOTIC BAYES ANALYSIS FOR THE FINITE-HORIZON ONE-ARMED-BANDIT PROBLEM
Burnetas, A., & Katehakis, M. N. (2003) · Probability in the Engineering and Informational Sciences · doi:10.1017/s0269964803171045 · Cited by 67
Toward a classification of finite partial-monitoring games
Antos, A., Bartók, G., Pál, D., & Szepesvári, C. (2012) · Theoretical Computer Science · doi:10.1016/j.tcs.2012.10.008 · Cited by 66
Self-citationNormal bandits of unknown means and variances
Cowan, W., Honda, J., & Katehakis, M. N. (2017) · Journal of Machine Learning Research · doi:10.5555/3122009.3242011 · Cited by 64
The Many Faces of Exponential Weights in Online LearningPreprint
van der Hoeven, D., van Erven, T., & Kotłowski, W. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.07543 · Cited by 63
On multilabel classification and ranking with bandit feedback
Gentile, C., & Orabona, F. (2014) · Cited by 62
Set-valued approachability and online learning with partial monitoring
Mannor, S., Perchet, V., & Stoltz, G. (2014) · Journal of Machine Learning Research · Cited by 62
Bandit Convex Optimization with Biased Noisy Gradient Oracles
Hu, X. (2017) · ERA: Education and Research Archive (University of Alberta) · doi:10.7939/r3jm23t75 · Cited by 62
On Thompson Sampling and Asymptotic Optimality
Leike, J., Lattimore, T., Orseau, L., & Hütter, M. (2017) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2017/688 · Cited by 61
Self-citationMax K-Armed Bandit: On the ExtremeHunter Algorithm and Beyond
Achab, M., Clémençon, S., Garivier, A., Sabourin, A., & Vernade, C. (2017) · Lecture notes in computer science · doi:10.1007/978-3-319-71246-8_24 · Cited by 61
Combinatorial Pure Exploration with Continuous and Separable Reward Functions and Its Applications
Huang, W., Ok, J., Li, L., & Chen, W. (2018) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2018/317 · Cited by 61
Martingales in Sequential Analysis and Time Series, 1945-1985 ∗
Lai, T. L. (2009) · Cited by 57
But you have to remember P.J. Daniell of Sheffield
Aldrich, J. (2007) · ePrints Soton (University of Southampton) · Cited by 57
Efficient-UCBV: An Almost Optimal Algorithm Using Variance Estimates
Mukherjee, S., Naveen, K. P., Sudarsanam, N., & Ravindran, B. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.12110 · Cited by 57
Lower Bounds and Selectivity of Weak-Consistent Policies in Stochastic Multi-Armed Bandit Problem
Salomon, A., Audibert, J.-Y., Alaoui, I. E., & Cesa-bianchi, N. (2013) · Cited by 54
Phased Exploration with Greedy Exploitation in Stochastic Combinatorial Partial Monitoring GamesPreprint
Chaudhuri, S., & Tewari, A. (2016) · arXiv (Cornell University) · doi:10.13140/rg.2.1.3252.9529 · Cited by 52
Cited by
Degenerate Feedback Loops in Recommender Systems
Jiang, R., Chiappa, S., Lattimore, T., György, A., & Kohli, P. (2019) · Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3306618.3314288 · Cited by 232
Self-citationA Survey of Reinforcement Learning from Human Feedback
Kaufmann, T., Weng, P., Bengs, V., & Hüllermeier, E. (2023) · arXiv · doi:10.48550/arxiv.2312.14925 · Cited by 43
Kadi, H. A., & Terzić, K. (2023) · Sensors · doi:10.3390/s23052389 · Cited by 27
“Some fundamental myopic exploration strategies are derived from the Multi-armed Bandit (MAB) formulation [196], then scaled to more complex MDP/POMDP settings.”
MethodConformal Decision Theory: Safe Autonomous Decisions from Imperfect Predictions
Lekeufack, J., Angelopoulos, A. N., Bajcsy, A., Jordan, M. I., & Malik, J. (2024) · 2024 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra57147.2024.10610041 · Cited by 18
Sublinear regret for learning POMDPs
Xiong, Y., Chen, N., Gao, X., & Zhou, X. (2022) · Production and Operations Management · doi:10.1111/poms.13778 · Cited by 6
Metadata
- Authors
- Lattimore, T., & Szepesvári, C.
- Venue
- Cambridge University Press eBooks
- Publisher
- Cambridge University Press
- Published
- 2020-07-04
- OpenAlex
- W4206530644
- Cited by
- 976
- References
- 244
- FWCI
- 55.96
- Type
- book
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.