The DOI matches its Crossref record; 204 of the 204 references it cites were found in a registry or an index.
Introduction to Multi-Armed Bandits
Open accessAbstract
Multi-armed bandits a simple but very powerful framework for algorithms that make decisions over time under uncertainty. An enormous body of work has accumulated over the years, covered in several books and surveys. This book provides a more introductory, textbook-like treatment of the subject. Each chapter tackles a particular line of work, providing a self-contained, teachable technical introduction and a brief review of the further developments.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Elements of Information Theory
Cover, T. M., & Thomas, J. A. (2001) · Wiley eBooks · doi:10.1002/0471200611 · Cited by 36,808
A decision-theoretic generalization of on-line learning and an application to boosting
Freund, Y., & Schapire, R. E. (1997) · Journal of Computer and System Sciences · doi:10.1006/jcss.1997.1504 · Cited by 20,466
Finite-time Analysis of the Multiarmed Bandit Problem
Auer, P., Cesa-Bianchi, N., & Fischer, P. (2002) · Machine Learning · doi:10.1023/a:1013689704352 · Cited by 6,033
Prediction, Learning, and Games
Cesa-Bianchi, N., & Lugosi, G. (2006) · Cambridge University Press eBooks · doi:10.1017/cbo9780511546921 · Cited by 3,162
Bandit Based Monte-Carlo Planning
Kocsis, L., & Szepesvári, C. (2006) · Lecture Notes in Computer Science · doi:10.1007/11871842_29 · Cited by 2,855
THOMPSON, W. R. (1933) · Biometrika · doi:10.1093/biomet/25.3-4.285 · Cited by 2,712
A contextual-bandit approach to personalized news article recommendation
Li, L., Chu, W., Langford, J., & Schapire, R. E. (2010) · Proceedings of the 19th international conference on World wide web · doi:10.1145/1772690.1772758 · Cited by 2,562
Asymptotically efficient adaptive allocation rules
Lai, T. L., & Robbins, H. (1985) · Advances in Applied Mathematics · doi:10.1016/0196-8858(85)90002-8 · Cited by 2,406
(2002) · doi:10.1137/s0097539701398375 · Cited by 2,275
Competition and innovation: an inverted U relationship
Blundell, R., Howitt, P., Griffith, R., Aghion, P., & Bloom, N. (2002) · Working Paper Series · doi:10.1920/wp.ifs.2002.0204 · Cited by 2,109
The Theory of Incentives: The Principal-Agent Model
Laffont, J., & Martimort, D. (2001) · Toulouse 1 Capitole Publications (Université Toulouse I Capitole) · Cited by 2,049
The Weighted Majority Algorithm
Littlestone, N., & Warmuth, M. K. (1994) · Information and Computation · doi:10.1006/inco.1994.1009 · Cited by 1,975
Information-Theoretic Regret Bounds for Gaussian Process Optimization in the Bandit Setting
Srinivas, N., Krause, A., Kakade, S. M., & Seeger, M. W. (2012) · IEEE Transactions on Information Theory · doi:10.1109/tit.2011.2182033 · Cited by 1,850
Subjectivity and Correlation in Randomized Strategies
Aumann, R. J. (1974) · Journal of Mathematical Economics · doi:10.1016/0304-4068(74)90037-8 · Cited by 1,630
Reinforcement Learning: An Introduction
(1998) · Kybernetes · doi:10.1108/k.1998.27.9.1093.3 · Cited by 1,472
Regret Analysis of Stochastic and Nonstochastic Multi-armed Bandit Problems
Bubeck, S. (2012) · now publishers, Inc. eBooks · doi:10.1561/9781601986276 · Cited by 1,404
Regret Analysis of Stochastic and Nonstochastic Multi-armed Bandit Problems
Bubeck, S., & Cesa-Bianchi, N. (2012) · Foundations and Trends® in Machine Learning · doi:10.1561/2200000024 · Cited by 1,295
Using confidence bounds for exploitation-exploration trade-offs
Auer, P. L. (2003) · Journal of Machine Learning Research · doi:10.5555/944919.944941 · Cited by 1,229
Algorithms for Reinforcement Learning
Szepesvári, C. (2010) · Synthesis Lectures on Artificial Intelligence and Machine Learning · doi:10.1007/978-3-031-01551-9 · Cited by 1,049
Multi-Armed Bandit Allocation Indices.
Bather, J. A., & Gittins, J. C. (1990) · Journal of the Royal Statistical Society. Series A (Statistics in Society) · doi:10.2307/2982809 · Cited by 998
Concentration of Measure for the Analysis of Randomized Algorithms
Dubhashi, D. P., & Panconesi, A. (2009) · Cambridge University Press eBooks · doi:10.1017/cbo9780511581274 · Cited by 945
Improved Algorithms for Linear Stochastic Bandits
Abbasi-Yadkori, Y., Pál, D., & Szepesvári, C. (2011) · Cited by 919
An Iterative Method of Solving a Game
Robinson, J. (1951) · The Annals of Mathematics · doi:10.2307/1969530 · Cited by 868
Analysis of Thompson Sampling for the multi-armed bandit problem
Agrawal, S., & Goyal, N. (2011) · arXiv (Cornell University) · doi:10.48550/arxiv.1111.1797 · Cited by 819
Introduction to Online Convex Optimization
Hazan, E. (2016) · now publishers, Inc. eBooks · doi:10.1561/9781680831719 · Cited by 813
On the Complexity of Best Arm Identification in Multi-Armed Bandit Models
Kaufmann, E., Cappé, O., & Garivier, A. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1407.4443 · Cited by 661
Contextual Bandits with Linear Payoff Functions
Chu, W., Li, L. C., Reyzin, L., & Schapire, R. E. (2011) · Cited by 633
Efficient algorithms for online decision problems
Kalai, A., & Vempala, S. (2005) · Journal of Computer and System Sciences · doi:10.1016/j.jcss.2004.10.016 · Cited by 591
(2008) · ScholarlyCommons (University of Pennsylvania) · Cited by 581
Bandit Problems, Sequential Allocation of Experiments.
Jones, P. W., Berry, D. A., & Fristedt, B. (1987) · The Journal of the Operational Research Society · doi:10.2307/2582852 · Cited by 566
Online convex optimization in the bandit setting: gradient descent without a gradient
Flaxman, A. D., Kalai, A. T., & McMahan, H. B. (2004) · arXiv (Cornell University) · doi:10.48550/arxiv.cs/0408007 · Cited by 546
Information Design: A Unified Perspective
Bergemann, D., & Morris, S. (2019) · Journal of Economic Literature · doi:10.1257/jel.20181489 · Cited by 527
Adaptive Game Playing Using Multiplicative Weights
Freund, Y., & Schapire, R. E. (1999) · Games and Economic Behavior · doi:10.1006/game.1999.0738 · Cited by 515
Dynamic Pricing Without Knowing the Demand Function: Risk Bounds and Near-Optimal Algorithms
Besbes, O., & Zeevi, A. (2009) · Operations Research · doi:10.1287/opre.1080.0640 · Cited by 506
Learning diverse rankings with multi-armed bandits
Radlinski, F., Kleinberg, R., & Joachims, T. (2008) · Proceedings of the 25th international conference on Machine learning - ICML '08 · doi:10.1145/1390156.1390255 · Cited by 503
Learning to Optimize via Posterior Sampling
Russo, D., & Van Roy, B. (2014) · Mathematics of Operations Research · doi:10.1287/moor.2014.0650 · Cited by 493
Adaptive design methods in clinical trials–a review
Chow, S.-C., & Chang, M. (2008) · Orphanet Journal of Rare Diseases · doi:10.1186/1750-1172-3-11 · Cited by 482
Dynamic Assortment Optimization with a Multinomial Logit Choice Model and Capacity Constraint
Rusmevichientong, P., Shen, Z. M., & Shmoys, D. B. (2010) · Operations Research · doi:10.1287/opre.1100.0866 · Cited by 474
Preventing Fairness Gerrymandering: Auditing and Learning for Subgroup Fairness
Kearns, M., Neel, S., Roth, A., & Wu, Z. S. (2017) · arXiv · doi:10.48550/arxiv.1711.05144 · Cited by 469 · cited 2×
“Second, repeated Lagrange games, in conjunction with regret minimization in zero-sum games, have been used as an algorithmic tool to solve convex optimization problems (different from BwK), application domains range from differential privacy to algorithmic fairness to learning from revealed preferences (Rogers et al.,…”
BackgroundMethodBest Arm Identification in Multi-Armed Bandits
Audibert, J.-Y., Bubeck, S., & Munos, R. (2010) · HAL · Cited by 446
Unbiased offline evaluation of contextual-bandit-based news article recommendation algorithms
Li, L., Chu, W., Langford, J., & Wang, X. (2011) · Proceedings of the fourth ACM international conference on Web search and data mining · doi:10.1145/1935826.1935878 · Cited by 443
On Upper-Confidence Bound Policies for Switching Bandit Problems
Garivier, A., & Moulines, E. (2011) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-24412-4_16 · Cited by 442
Linearly Parameterized Bandits
Rusmevichientong, P., & Tsitsiklis, J. N. (2010) · Mathematics of Operations Research · doi:10.1287/moor.1100.0446 · Cited by 427
Dynamic pricing and learning: Historical origins, current research, and new directions
den Boer, A. V. (2015) · Surveys in Operations Research and Management Science · doi:10.1016/j.sorms.2015.03.001 · Cited by 427
Bounded geometries, fractals, and low-distortion embeddings
Gupta, A., Krauthgamer, R., & Lee, J. R. (2004) · 44th Annual IEEE Symposium on Foundations of Computer Science, 2003. Proceedings. · doi:10.1109/sfcs.2003.1238226 · Cited by 405
Bayesian Persuasion and Information Design
Kamenica, E. (2019) · Annual Review of Economics · doi:10.1146/annurev-economics-080218-025739 · Cited by 404
Multi-armed bandits in metric spaces
Kleinberg, R., Slivkins, A., & Upfal, E. (2008) · ACM Symposium on Theory of Computing (STOC) · doi:10.1145/1374376.1374475 · Cited by 396
Self-citationCombinatorial Multi-Armed Bandit: General Framework and Applications
Chen, W., Wang, Y., & Yang, Y. (2013) · International Conference on Machine Learning · Cited by 392
Distributed Learning in Multi-Armed Bandit With Multiple Players
Liu, K., & Zhao, Q. (2010) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2010.2062509 · Cited by 385
Game theory, on-line prediction and boosting
Freund, Y., & Schapire, R. E. (1996) · Proceedings of the ninth annual conference on Computational learning theory - COLT '96 · doi:10.1145/238061.238163 · Cited by 383
A Tutorial on Thompson Sampling
Russo, D. J., Van Roy, B., Kazerouni, A., Osband, I., & Wen, Z. (2018) · now publishers, Inc. eBooks · doi:10.1561/9781680834710 · Cited by 380
Calibrated Learning and Correlated Equilibrium
Foster, D. P., & Vohra, R. V. (1997) · Games and Economic Behavior · doi:10.1006/game.1997.0595 · Cited by 365
Thompson Sampling: An Asymptotically Optimal Finite-Time Analysis
Kaufmann, E., Korda, N., & Munos, R. (2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-34106-9_18 · Cited by 365
Parallelizing Exploration-Exploitation Tradeoffs with Gaussian Process Bandit OptimizationPreprint
Desautels, T., Krause, A., & Burdick, J. W. (2012) · arXiv (Cornell University) · doi:10.48550/arxiv.1206.6402 · Cited by 353
Finding nearest neighbors in growth-restricted metrics
Karger, D. R., & Ruhl, M. (2002) · Proceedings of the thiry-fourth annual ACM symposium on Theory of computing · doi:10.1145/509907.510013 · Cited by 334
The Epoch-Greedy algorithm for contextual multi-armed bandits
Langford, J. C., & Zhang, T. (2007) · Cited by 320
Intrinsic robustness of the price of anarchy
Roughgarden, T. (2009) · Proceedings of the forty-first annual ACM symposium on Theory of computing · doi:10.1145/1536414.1536485 · Cited by 319
Cesa-Bianchi, N., & Lugosi, G. (2012) · Journal of Computer and System Sciences · doi:10.1016/j.jcss.2012.01.001 · Cited by 319
The Sample Complexity of Exploration in the Multi-Armed Bandit Problem
Mannor, S., & Tsitsiklis, J. N. (2004) · doi:10.5555/1005332.1005355 · Cited by 307
Dynamic Assortment with Demand Learning for Seasonal Consumer Goods
Caro, F., & Gallien, J. (2007) · Management Science · doi:10.1287/mnsc.1060.0613 · Cited by 306
Regret in the On-Line Decision Problem
Foster, D. P., & Vohra, R. (1999) · Games and Economic Behavior · doi:10.1006/game.1999.0740 · Cited by 302
PAC Bounds for Multi-armed Bandit and Markov Decision Processes
Even-Dar, E., Mannor, S., & Mansour, Y. (2002) · Lecture Notes in Computer Science · doi:10.1007/3-540-45435-7_18 · Cited by 298
Nearly Tight Bounds for the Continuum-Armed Bandit Problem
Kleinberg, R. (2004) · Cited by 294
The value of knowing a demand curve: bounds on regret for online posted-price auctions
Kleinberg, R., & Leighton, T. (2004) · 44th Annual IEEE Symposium on Foundations of Computer Science, 2003. Proceedings. · doi:10.1109/sfcs.2003.1238232 · Cited by 290
Further Optimal Regret Bounds for Thompson Sampling
Agrawal, S., & Goyal, N. (2012) · arXiv (Cornell University) · doi:10.48550/arxiv.1209.3353 · Cited by 287
Collaborative Filtering Bandits
Li, S., Karatzoglou, A., & Gentile, C. (2016) · Proceedings of the 39th International ACM SIGIR conference on Research and Development in Information Retrieval · doi:10.1145/2911451.2911548 · Cited by 287
Truthful incentives in crowdsourcing tasks using regret minimization mechanisms
Singla, A. K., & Krause, A. (2013) · International World Wide Web Conference (WWW) · doi:10.1145/2488388.2488490 · Cited by 276
Taming the Monster: A Fast and Simple Algorithm for Contextual Bandits
Agarwal, A., Hsu, D., Kale, S., Langford, J. C., Li, L. C., & Schapire, R. E. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1402.0555 · Cited by 273
Interactively optimizing information retrieval systems as a dueling bandits problem
Yue, Y., & Joachims, T. (2009) · Proceedings of the 26th Annual International Conference on Machine Learning · doi:10.1145/1553374.1553527 · Cited by 268
Contextual Gaussian Process Bandit Optimization
Krause, A., & Ong, C. S. (2011) · Cited by 267
Implementing the “Wisdom of the Crowd”
Kremer, I., Mansour, Y., & Perry, M. (2014) · Journal of Political Economy · doi:10.1086/676597 · Cited by 259
PCC: Re-architecting Congestion Control for Consistent High Performance
Dong, M., Li, Q., Zarchy, D., Godfrey, B., & Schapira, M. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1409.7092 · Cited by 237
An Online Convex Optimization Approach to Proactive Network Resource Allocation
Chen, T., Ling, Q., & Giannakis, G. B. (2017) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2017.2750109 · Cited by 236
Using and combining predictors that specialize
Freund, Y., Schapire, R. E., Singer, Y., & Warmuth, M. K. (1997) · Proceedings of the twenty-ninth annual ACM symposium on Theory of computing - STOC '97 · doi:10.1145/258533.258616 · Cited by 234
Munos, R. (2014) · Foundations and Trends® in Machine Learning · doi:10.1561/2200000038 · Cited by 233
Contextual Bandits with Similarity Information
Slivkins, A. (2009) · arXiv (Cornell University) · doi:10.48550/arxiv.0907.3986 · Cited by 232
Self-citationMoulin, H., & Vial, J. P. (1978) · International Journal of Game Theory · doi:10.1007/bf01769190 · Cited by 227
The K-armed dueling bandits problem
Yue, Y., Broder, J., Kleinberg, R., & Joachims, T. (2012) · Journal of Computer and System Sciences · doi:10.1016/j.jcss.2011.12.028 · Cited by 212
Regret Bounds and Minimax Policies under Partial Monitoring
Audibert, J.-Y., & Bubeck, S. (2010) · HAL · Cited by 210
An Efficient Dynamic Mechanism
(2013) · Econometrica · doi:10.3982/ecta6995 · Cited by 210
Mostly Exploration-Free Algorithms for Contextual Bandits
Bastani, H., Bayati, M., & Khosravi, K. (2020) · Management Science · doi:10.1287/mnsc.2020.3605 · Cited by 209
Bypassing the embedding: algorithms for low dimensional metrics
Talwar, K. (2004) · Proceedings of the thirty-sixth annual ACM symposium on Theory of computing · doi:10.1145/1007352.1007399 · Cited by 207
Making Contextual Decisions with Low Technical DebtPreprint
Agarwal, A., Bird, S., Cozowicz, M., Hoang, L., Langford, J. C., Lee, S., Li, J., Melamed, D., Oshri, G., Ribas, O., et al. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1606.03966 · Cited by 207
Self-citationRobust Predictions in Games With Incomplete Information
(2013) · Econometrica · doi:10.3982/ecta11105 · Cited by 206
The Continuum-Armed Bandit Problem
Agrawal, R. (1995) · SIAM Journal on Control and Optimization · doi:10.1137/s0363012992237273 · Cited by 200
Regret bounds for sleeping experts and bandits
Kleinberg, R., Niculescu-Mizil, A., & Sharma, Y. (2010) · Machine Learning · doi:10.1007/s10994-010-5178-7 · Cited by 198
Batch learning from logged bandit feedback through counterfactual risk minimization
Swaminathan, A., & Joachims, T. (2015) · Journal of Machine Learning Research · Cited by 193
Blum, A. (1997) · Machine Learning · doi:10.1023/a:1007335615132 · Cited by 191
Trading Regret for Efficiency: Online Convex Optimization with Long Term ConstraintsPreprint
MahdaviMehrdad, JinRong, & YangTianbao (2011) · arXiv (Cornell University) · doi:10.48550/arxiv.1111.6082 · Cited by 186
Close the Gaps: A Learning-While-Doing Algorithm for Single-Product Revenue Management Problems
Wang, Z., Deng, S., & Ye, Y. (2014) · Operations Research · doi:10.1287/opre.2013.1245 · Cited by 183
An Online Algorithm for Maximizing Submodular Functions
Streeter, M., & Golovin, D. (2007) · doi:10.21236/ada476748 · Cited by 183
Optimal Dynamic Assortment Planning with Demand Learning
Sauré, D., & Zeevi, A. (2013) · Manufacturing & Service Operations Management · doi:10.1287/msom.2013.0429 · Cited by 181
Improved Rates for the Stochastic Continuum-Armed Bandit Problem
Auer, P. L., Ortner, R., & Szepesvári, C. (2007) · Springer eBooks · doi:10.1007/978-3-540-72927-3_33 · Cited by 178
Cascading Bandits: Learning to Rank in the Cascade ModelPreprint
Kveton, B., Szepesvári, C., Wen, Z., & Ashkan, A. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1502.02763 · Cited by 178
Online linear optimization and adaptive routing
Awerbuch, B., & Kleinberg, R. (2007) · Journal of Computer and System Sciences · doi:10.1016/j.jcss.2007.04.016 · Cited by 177
Contextual Bandit Algorithms with Supervised Learning Guarantees
Beygelzimer, A., Langford, J. C., Li, L. C., Reyzin, L., & Schapire, R. E. (2010) · arXiv (Cornell University) · doi:10.48550/arxiv.1002.4058 · Cited by 175
Doubly Robust Policy Evaluation and Optimization
Dudı́k, M., Erhan, D., Langford, J. C., & Li, L. (2014) · Cited by 174
Online Geometric Optimization in the Bandit Setting Against an Adaptive Adversary
McMahan, H. B., & Blum, A. (2004) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-27819-1_8 · Cited by 172
Blind Network Revenue Management
Besbes, O., & Zeevi, A. (2012) · Operations Research · doi:10.1287/opre.1120.1103 · Cited by 171
Bandit Algorithms for Tree Search
Coquelin, P.-A., & Munos, R. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1408.2028 · Cited by 170
Knapsack Based Optimal Policies for Budget–Limited Multi–Armed Bandits
Tran-Thanh, L., Chapman, A. C., Rogers, A., & Jennings, N. R. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v26i1.8279 · Cited by 167
Tight Regret Bounds for Stochastic Combinatorial Semi-BanditsPreprint
Kveton, B., Wen, Z., Ashkan, A., & Szepesvári, C. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1410.0949 · Cited by 165
Recommender Systems as Mechanisms for Social Learning*
Che, Y.-K., & Hörner, J. (2018) · The Quarterly Journal of Economics · doi:10.1093/qje/qjx044 · Cited by 163
From External to Internal Regret
Blum, A. L., & Monsour, Y. (2018) · Figshare · doi:10.1184/r1/6605876 · Cited by 163
The best of both worlds: stochastic and adversarial banditsPreprint
Bubeck, S., & Slivkins, A. (2012) · arXiv (Cornell University) · doi:10.48550/arxiv.1202.4473 · Cited by 162
Self-citationBadanidiyuru, A., Kleinberg, R., & Slivkins, A. (2018) · Journal of the ACM · doi:10.1145/3164539 · Cited by 162
Self-citationMulti-armed bandit problems with dependent arms
Pandey, S., Chakrabarti, D., & Agarwal, D. (2007) · Proceedings of the 24th international conference on Machine learning · doi:10.1145/1273496.1273587 · Cited by 161
MNL-Bandit: A Dynamic Learning Approach to Assortment Selection
Agrawal, S., Avadhanula, V., Goyal, V., & Zeevi, A. J. (2019) · Operations Research · doi:10.1287/opre.2018.1832 · Cited by 157
Learning in Repeated Auctions with Budgets: Regret Minimization and Equilibrium
Balseiro, S., & Gur, Y. (2017) · Management Science · doi:10.1287/mnsc.2018.3174 · Cited by 154
Optimistic Optimization of a Deterministic Function without the Knowledge of its Smoothness
Munos, R. (2011) · Cited by 151
Online Bandit Learning against an Adaptive Adversary: from Regret to Policy RegretPreprint
Arora, R., Dekel, O., & Tewari, A. (2012) · arXiv (Cornell University) · doi:10.48550/arxiv.1206.6400 · Cited by 150
Adapting to a Changing Environment: the Brownian Restless Bandits.
Slivkins, A., & Upfal, E. (2008) · Cited by 144
Self-citationOnline Evaluation for Information Retrieval
Hofmann, K., Li, L. C., & Radlinski, F. (2016) · now publishers, Inc. eBooks · doi:10.1561/1500000051 · Cited by 143
Online Convex Optimization with Time-Varying ConstraintsPreprint
Neely, M. J., & Yu, H. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1702.04783 · Cited by 143
Near-Optimal Regret Bounds for Thompson Sampling
Agrawal, S., & Goyal, N. (2017) · Journal of the ACM · doi:10.1145/3088510 · Cited by 140
Bandit Convex Optimization for Scalable and Dynamic IoT Management
Chen, T., & Giannakis, G. B. (2019) · IEEE Internet of Things Journal · doi:10.1109/jiot.2018.2839563 · Cited by 139
An Asymptotically Optimal Bandit Algorithm for Bounded Support Models.
Honda, J., & Akimichi Takemura, Y. (2010) · Cited by 139
Bandits with concave rewards and convex knapsacks
Agrawal, S., & Devanur, N. R. (2014) · ACM Conference on Economics and Computation (EC) · doi:10.1145/2600057.2602844 · Cited by 138
Bergemann, D., & Välimäki, J. (2000) · The Review of Economic Studies · doi:10.1111/1467-937x.00128 · Cited by 134
Mechanism Design via Correlation Gap
Yan, Q. (2011) · ACM-SIAM Symposium on Discrete Algorithms (SODA) · doi:10.1137/1.9781611973082.56 · Cited by 134
Multiplicative updates outperform generic no-regret learning in congestion games
Kleinberg, R., Piliouras, G., & Eva Tardos, D. (2009) · ACM Symposium on Theory of Computing (STOC) · doi:10.1145/1536414.1536487 · Cited by 133
Reinforcement Learning with Immediate Rewards and Linear Hypotheses
Abe, N., Biermann, A., & Long, P. M. (2003) · Algorithmica · doi:10.1007/s00453-003-1038-1 · Cited by 133
Stochastic bandits robust to adversarial corruptions
Lykouris, T., Mirrokni, V., & Leme, R. P. (2018) · ACM Symposium on Theory of Computing (STOC) · doi:10.1145/3188745.3188918 · Cited by 133
Multi-Armed Bandit with Budget Constraint and Variable Costs
Ding, W., Qin, T., Zhang, X., & Liu, T. (2013) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v27i1.8637 · Cited by 131
Chakrabarti, D., Kumar, R., Radlinski, F., & Upfal, E. (2008) · Cited by 126
A Finite-Time Analysis of Multi-armed Bandits Problems with Kullback-Leibler DivergencesPreprint
Maillard, O.-A., Munos, R., & Stoltz, G. (2011) · arXiv (Cornell University) · doi:10.48550/arxiv.1105.5820 · Cited by 124
Cycles in adversarial regularized learning
Mertikopoulos, P., Papadimitriou, C., & Piliouras, G. (2018) · Proceedings of the Twenty-Ninth Annual ACM-SIAM Symposium on Discrete Algorithms · doi:10.1137/1.9781611975031.172 · Cited by 122
The price of truthfulness for pay-per-click auctions
Devanur, N. R., & Kakade, S. M. (2009) · ACM Conference on Electronic Commerce (EC) · doi:10.1145/1566374.1566388 · Cited by 121
Partial Monitoring—Classification, Regret Bounds, and Algorithms
Bartók, G., Foster, D. P., Pál, D., Rakhlin, A., & Szepesvári, C. (2014) · Mathematics of Operations Research · doi:10.1287/moor.2014.0663 · Cited by 118
Bandits and Experts in Metric Spaces
Kleinberg, R., Slivkins, A., & Upfal, E. (2019) · Journal of the ACM · doi:10.1145/3299873 · Cited by 111
Self-citationFrazier, P. I., Kempe, D., Kleinberg, J. M., & Kleinberg, R. (2014) · ACM Conference on Economics and Computation (EC) · doi:10.1145/2600057.2602897 · Cited by 110
A Contextual Bandit Bake-offPreprint
Bietti, A., Agarwal, A., & Langford, J. C. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.04064 · Cited by 108
Learning Prices for Repeated Auctions with Strategic BuyersPreprint
Amin, K., Rostamizadeh, A., & Syed, U. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1311.6838 · Cited by 105
Dynamic pricing with limited supply
Babaioff, M., Dughmi, S., Kleinberg, R., & Slivkins, A. (2012) · ACM Conference on Electronic Commerce (EC) · doi:10.1145/2229012.2229023 · Cited by 105
Self-citationKernel-based Methods for Bandit Convex Optimization
Bubeck, S., Lee, Y. T., & Eldan, R. (2017) · ACM Symposium on Theory of Computing (STOC) · doi:10.1145/3055399.3055403 · Cited by 104
Reducing Dueling Bandits to Cardinal BanditsPreprint
Ailon, N., Joachims, T., & Karnin, Z. S. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1405.3396 · Cited by 103
Lipschitz Bandits: Regret Lower Bound and Optimal Algorithms
Magureanu, S., Combes, R., & Proutière, A. (2014) · Conference on Learning Theory · Cited by 99
Unimodal Bandits: Regret Lower Bounds and Optimal AlgorithmsPreprint
Combes, R., & Proutière, A. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1405.5096 · Cited by 98
Via: Improving Internet Telephony Call Quality Using Predictive Relay Selection
Jiang, J., Das, R., Ananthanarayanan, G., Chou, P. A., Padmanabhan, V. N., Sekar, V., Dominique, E., Goliszewski, M., Kukoleca, D., Vafin, R., et al. (2016) · ACM SIGCOMM Conference (SIGCOMM) · doi:10.1145/2934872.2934907 · Cited by 97
Efficient Contextual Bandits in Non-stationary WorldsPreprint
Luo, H., Wei, C.-Y., Agarwal, A., et al. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1708.01799 · Cited by 97
Jiang, J., Sun, Y. S., Sekar, V., & Zhang, H. (2017) · Networked Systems Design and Implementation · Cited by 96
Multiplicative Weights Update in Zero-Sum Games
Bailey, J. P., & Piliouras, G. (2018) · ACM Conference on Economics and Computation (EC) · doi:10.1145/3219166.3219235 · Cited by 95
Matroid Bandits: Fast Combinatorial Optimization with LearningPreprint
Kveton, B., Wen, Z., Ashkan, A., Eydgahi, H., & Eriksson, B. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1403.5045 · Cited by 95
SOME NOTES ON COMPUTATION OF GAMES SOLUTIONS
Brown, G. W. (1949) · Defense Technical Information Center (DTIC) · Cited by 91
Concurrent Bandits and Cognitive Radio Networks
Avner, O., & Mannor, S. (2014) · Lecture notes in computer science · doi:10.1007/978-3-662-44848-9_5 · Cited by 87
Ho, C.-J., Slivkins, A., & Vaughan, J. W. (2016) · Journal of Artificial Intelligence Research · doi:10.1613/jair.4940 · Cited by 87
Self-citationOne Practical Algorithm for Both Stochastic and Adversarial Bandits
Seldin, Y., & Slivkins, A. (2014) · QUT ePrints (Queensland University of Technology) · Cited by 84
Self-citationε-first policies for budget-limited multi-armed bandits
Han, T. A., Chapman, A. C., de Cote, E. M., Rogers, A., & Jennings, N. R. (2010) · Spiral (Imperial College London) · Cited by 83
Counterfactual Estimation and Optimization of Click Metrics in Search Engines
Li, L., Chen, S., Kleban, J., & Gupta, A. (2015) · International World Wide Web Conference (WWW) · doi:10.1145/2740908.2742562 · Cited by 82
Regret and Convergence Bounds for a Class of Continuum-Armed Bandit Problems
Cope, E. W. (2009) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2009.2019797 · Cited by 79
Stochastic simultaneous optimistic optimizationPreprint
Valko, M., Carpentier, A., & Munos, R. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2604.24537 · Cited by 76
Prior-free and prior-dependent regret bounds for Thompson Sampling
Bubeck, S., & Liu, C.-Y. (2014) · Conference on Information Sciences and Systems · doi:10.1109/ciss.2014.6814158 · Cited by 76
Algorithms with Logarithmic or Sublinear Regret for Constrained Contextual BanditsPreprint
Wu, H., Srikant, R., Liu, X., & Jiang, C. W. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1504.06937 · Cited by 74
Golub, B., & Sadler, E. D. (2016) · doi:10.1093/oxfordhb/9780199948277.013.12 · Cited by 74
Lipschitz Bandits without the Lipschitz Constant
Bubeck, S., Stoltz, G., & Yu, J. Y. (2011) · Lecture notes in computer science · doi:10.1007/978-3-642-24412-4_14 · Cited by 72
Online decision making in crowdsourcing markets: theoretical challenges
Slivkins, A., & Vaughan, J. W. (2014) · ACM SIGecom Exchanges · doi:10.1145/2692359.2692364 · Cited by 69
Self-citationRelative Upper Confidence Bound for the K-Armed Dueling Bandit Problem
Zoghi, M., Whiteson, S., Munos, R., & de Rijke, M. (2013) · UvA-DARE (University of Amsterdam) · doi:10.48550/arxiv.1312.3393 · Cited by 67
Toward a classification of finite partial-monitoring games
Antos, A., Bartók, G., Pál, D., & Szepesvári, C. (2012) · Theoretical Computer Science · doi:10.1016/j.tcs.2012.10.008 · Cited by 66
Dynamic Auctions: A SurveyPreprint
Bergemann, D., & Said, M. (2010) · SSRN Electronic Journal · doi:10.2139/ssrn.1576613 · Cited by 65
Combinatorial Cascading BanditsPreprint
Kveton, B., Wen, Z., Ashkan, A., & Szepesvári, C. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1507.04208 · Cited by 64
Non-Stochastic Bandit Slate Problems
Kale, S., Reyzin, L., & Schapire, R. E. (2010) · Cited by 62
A New Algorithm for Non-stationary Contextual Bandits: Efficient, Optimal, and Parameter-freePreprint
Chen, Y., Lee, C., Luo, H., & Wei, C.-Y. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1902.00980 · Cited by 62
An algorithm with nearly optimal pseudo-regret for both stochastic and adversarial banditsPreprint
Auer, P. L., & Chiang, C.-K. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1605.08722 · Cited by 58
Competing Bandits: Learning Under CompetitionPreprint
Mansour, Y., Slivkins, A., & Wu, Z. S. (2017) · arXiv (Cornell University) · doi:10.4230/lipics.itcs.2018.48 · Cited by 56
Self-citationAn efficient algorithm for contextual bandits with knapsacks, and an extension to concave objectivesPreprint
Agrawal, S., Devanur, N. R., & Li, L. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1506.03374 · Cited by 56
Triangulation and embedding using small sets of beacons
Kleinberg, J. M., Slivkins, A., & Wexler, T. (2009) · Journal of the ACM · doi:10.1145/1568318.1568322 · Cited by 54
Self-citationPractical Contextual Bandits with Regression OraclesPreprint
Foster, D. J., Agarwal, A., Dudík, M., Luo, H., & Schapire, R. E. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1803.01088 · Cited by 54
Black-box optimization of noisy functions with unknown smoothnessPreprint
Grill, J.-B., Valko, M., & Munos, R. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2605.02462 · Cited by 52
A truthful learning mechanism for contextual multi-slot sponsored search auctions with externalities
Gatti, N., Lazaric, A., & Trovò, F. (2012) · ACM Conference on Electronic Commerce (EC) · doi:10.1145/2229012.2229057 · Cited by 51
Online learning in repeated auctionsPreprint
Weed, J., Perchet, V., & Rigollet, P. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1511.05720 · Cited by 51
Learning to Bid Without Knowing your Value
Feng, Z., Podimata, C., & Syrgkanis, V. (2018) · ACM Conference on Economics and Computation (EC) · doi:10.1145/3219166.3219208 · Cited by 49
Characterizing Truthful Multi-armed Bandit Mechanisms
Babaioff, M., Sharma, Y., & Slivkins, A. (2014) · SIAM Journal on Computing · doi:10.1137/120878768 · Cited by 48
Self-citationContextual Bandit Learning with Predictable RewardsPreprint
Agarwal, A., Dudík, M., Kale, S., Langford, J. C., & Schapire, R. E. (2012) · arXiv (Cornell University) · doi:10.48550/arxiv.1202.1334 · Cited by 47
Near-optimal no-regret algorithms for zero-sum games
Daskalakis, C., Deckelbaum, A., & Kim, A. (2014) · Games and Economic Behavior · doi:10.1016/j.geb.2014.01.003 · Cited by 46
Seldin, Y., & Lugosi, G. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1702.06103 · Cited by 46
Braverman, M., Mao, J., Schneider, J., & Weinberg, S. M. (2018) · ACM Conference on Economics and Computation (EC) · doi:10.1145/3219166.3219233 · Cited by 44
Incentivizing Exploration with Selective Data Disclosure
Immorlica, N., Mao, J., Slivkins, A., & Wu, Z. S. (2020) · ACM Conference on Economics and Computation (EC) · doi:10.1145/3391403.3399487 · Cited by 44
Self-citationJointly Private Convex Programming
Hsu, J., Huang, Z., Roth, A., & Wu, Z. S. (2015) · ACM-SIAM Symposium on Discrete Algorithms (SODA) · doi:10.1137/1.9781611974331.ch43 · Cited by 43
Yu, J. Y., & Mannor, S. (2011) · International Conference on Machine Learning · Cited by 43
Adversarial Bandits with Knapsacks
Immorlica, N., Sankararaman, K. A., Schapire, R. E., & Slivkins, A. (2019) · Annual Symposium on Foundations of Computer Science · doi:10.1109/focs.2019.00022 · Cited by 43
Self-citationMulti-armed bandits on implicit metric spaces
Slivkins, A. (2011) · Cited by 42
Self-citationOnline Stochastic Optimization under Correlated Bandit FeedbackPreprint
Azar, M. G., Lazaric, A., & Brunskill, E. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1402.0562 · Cited by 42
Learning and incentives in user-generated content: multi-armed bandits with endogenous arms
Ghosh, A., & Hummel, P. (2013) · Conference on Innovations in Theoretical Computer Science · doi:10.1145/2422436.2422465 · Cited by 39
Bandits with Budgets: Regret Lower Bounds and Optimal Algorithms
Combes, R., Jiang, C. W., & Srikant, R. (2015) · ACM SIGMETRICS International Conference on Measurement and Modeling of Computer Systems (SIGMETRICS) · doi:10.1145/2745844.2745847 · Cited by 39
Better Algorithms for Benign Bandits
Hazan, E., & Kale, S. (2009) · ACM-SIAM Symposium on Discrete Algorithms (SODA) · doi:10.1137/1.9781611973068.5 · Cited by 33
Bayesian Incentive-Compatible Bandit Exploration
Mansour, Y., Slivkins, A., & Syrgkanis, V. (2020) · Operations Research · doi:10.1287/opre.2019.1949 · Cited by 33
Self-citationCheung, Y. K., & Piliouras, G. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.08396 · Cited by 32
Combinatorial Semi-Bandits with KnapsacksPreprint
Sankararaman, K. A., & Slivkins, A. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1705.08110 · Cited by 32
Self-citationBandits with Global Convex Constraints and Objective
Agrawal, S., & Devanur, N. R. (2019) · Operations Research · doi:10.1287/opre.2019.1840 · Cited by 31
Bull, A. D. (2015) · Bernoulli · doi:10.3150/14-bej644 · Cited by 30
Online Learning in Adversarial Lipschitz Environments
Maillard, O.-A., & Munos, R. (2010) · Lecture notes in computer science · doi:10.1007/978-3-642-15883-4_20 · Cited by 29
Provably competitive adaptive routing
Awerbuch, B., Holmer, D., Rubens, H., & Kleinberg, R. (2005) · IEEE Conference on Computer Communications (INFOCOM) · doi:10.1109/infcom.2005.1497929 · Cited by 29
Continuous Time Associative Bandit Problems ∗
György, A., Kocsis, L., Szabó, I., & Szepesvári, C. (2007) · Cited by 27
A Counter-example to Karlin's Strong Conjecture for Fictitious Play
Daskalakis, C., & Pan, Q. (2014) · Annual Symposium on Foundations of Computer Science · doi:10.1109/focs.2014.10 · Cited by 26
Dynamic Ad Allocation: Bandits with BudgetsPreprint
Slivkins, A. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1306.0155 · Cited by 25
Self-citationMultidimensional Dynamic Pricing for Welfare Maximization
Roth, A., Slivkins, A., Ullman, J., & Wu, Z. S. (2017) · ACM Conference on Economics and Computation (EC) · doi:10.1145/3033274.3085106 · Cited by 22
Self-citationLogarithmic regret bounds for Bandits with KnapsacksPreprint
Flajolet, A., & Jaillet, P. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1510.01800 · Cited by 21
Price Dispersion and Learning in a Dynamic Differentiated-Goods Duopoly
Keller, G., & Rady, S. (2001) · Econstor (Econstor) · doi:10.5282/ubm/epub.21 · Cited by 19
Bandits, Query Learning, and the Haystack Dimension
Amin, K., Kearns, M. J., & Syed, U. (2011) · Cited by 18
Inducing Approximately Optimal Flow Using Truthful Mediators
Rogers, R., Roth, A., Ullman, J., & Wu, Z. S. (2015) · ACM Conference on Economics and Computation (EC) · doi:10.1145/2764468.2764509 · Cited by 13
Cited by
Lattimore, T., & Szepesvári, C. (2020) · Cambridge University Press eBooks · doi:10.1017/9781108571401 · Cited by 976
A Survey of Beam Management for mmWave and THz Communications Towards 6G
Xue, Q., Ji, C., Ma, S., Guo, J., Xu, Y., Chen, Q., & Zhang, W. (2024) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2024.3361991 · Cited by 280
Reinforcement learning and bandits for speech and language processing: Tutorial, review and outlook
Lin, B. (2024) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.122254 · Cited by 28
Learning-Based Resource Management for Low-Power and Lossy IoT Networks
Musaddiq, A., Ali, R., Kim, S. W., & Kim, D.-S. (2022) · IEEE Internet of Things Journal · doi:10.1109/jiot.2022.3152929 · Cited by 23
A Sleeping, Recovering Bandit Algorithm for Optimizing Recurring Notifications
Yancey, K. P., & Settles, B. (2020) · Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining · doi:10.1145/3394486.3403351 · Cited by 5
Metadata
- Authors
- Slivkins, A.
- Venue
- Foundations and Trends® in Machine Learning
- Publisher
- Emerald
- Published
- 2019-11-08
- Volume
- 12
- Issue
- 1-2
- Pages
- 1-286
- ISSN
- 1935-8237 (ISSN-L)
- 1935-8245
- OpenAlex
- W2939627158
- Cited by
- 780
- References
- 204
- FWCI
- 47.40
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.