Finite-time Analysis of the Multiarmed Bandit Problem
Open accessThe claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Across all runs, the DGM-H independently creates variants of Upper Confidence Bound (UCB) style selection, combining normalized performance with explicit exploration bonuses.
“Across all runs, the DGM-H independently creates variants of Upper Confidence Bound (UCB) (Auer et al., 2002) style selection, combining normalized performance with explicit exploration bonuses:”
From Hyperagents (2026) · cites this paper as Auer et al., 2002
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225 · see citing works
Flagged first, then by citations
Research works printed but not linked: 7
- Duff, M. (1995). Q-learning for bandit problems. In Proceedings of the 12th International Conference on Machine Learning (pp. 209-217).
- Gittins, J. (1989). Multi-armed bandit allocation indices, Wiley-Interscience series in Systems and Optimization. New York: John Wiley and Sons.
- Holland, J. (1992). Adaptation in natural and artificial systems. Cambridge: MIT Press/Bradford Books.
- Ishikida, T., & Varaiya, P. (1994). Multi-armed bandit problem revisited. Journal of Optimization Theory and Applications, 83:1, 113-154.
- Pollard, D. (1984). Convergence of stochastic processes. Berlin: Springer.
- Wilks, S. (1962). Matematical statistics. New York: John Wiley and Sons.
- Yakowitz, S., & Lowe, W. (1991). Nonparametric bandit methods. Annals of Operations Research, 28, 297- 312. Received September 29, 2000 Revised May 21, 2001 Accepted June 20, 2001 Final manuscript June 20, 2001
Cited by
Sorted by citations
Deep Reinforcement Learning with Double Q-Learning
Van Hasselt, H., Guez, A., & Silver, D. (2016) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v30i1.10295 · Cited by 3,318 · see citing works
A Survey of Monte Carlo Tree Search Methods
Browne, C. B., Powley, E., Whitehouse, D., Lucas, S. M., Cowling, P. I., Rohlfshagen, P., Tavener, S., Perez, D., Samothrakis, S., & Colton, S. (2012) · IEEE Transactions on Computational Intelligence and AI in Games · doi:10.1109/tciaig.2012.2186810 · Cited by 2,970 · see citing works
A contextual-bandit approach to personalized news article recommendation
Li, L., Chu, W., Langford, J., & Schapire, R. E. (2010) · Proceedings of the 19th international conference on World wide web · doi:10.1145/1772690.1772758 · Cited by 2,562 · see citing works
“In contrast to theunguidedexploration strategy adopted by ǫgreedy, another class of algorithms generally known as upper confidence bound algorithms [4, 7, 17] use a smarter way to balanc e exploration and exploitation.”
BackgroundMethodInfluentialWhy Copy Others? Insights from the Social Learning Strategies Tournament
Rendell, L., Boyd, R., Cownden, D., Enquist, M., Eriksson, K., Feldman, M. W., Fogarty, L., Ghirlanda, S., Lillicrap, T., & Laland, K. N. (2010) · Science · doi:10.1126/science.1184719 · Cited by 811 · see citing works
Humans use directed and random exploration to solve the explore–exploit dilemma.
Wilson, R. C., Geana, A., White, J. M., Ludvig, E. A., & Cohen, J. D. (2014) · Journal of Experimental Psychology: General · doi:10.1037/a0038199 · Cited by 580 · see citing works
Li, Y., Jiang, Y., Li, Z., & Xia, S.-T. (2024) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2022.3182979 · Cited by 682 · see citing works
Learning to reinforcement learn
Wang, J. X., Kurth-Nelson, Z., Tirumala, D., Soyer, H., Leibo, J. Z., Munos, R., Blundell, C., Kumaran, D., & Botvinick, M. (2016) · arXiv · doi:10.48550/arxiv.1611.05763 · Cited by 579 · see citing works
A survey of real-time strategy game AI research and competition in StarCraft
Ontanon, S., Synnaeve, G., Uriarte, A., Richoux, F., Churchill, D., & Preuss, M. (2013) · IEEE Transactions on Computational Intelligence and AI in Games · doi:10.1109/tciaig.2013.2286295 · Cited by 423 · see citing works
Reinforcement learning for intelligent healthcare applications: A survey
Coronato, A., Naeem, M., De Pietro, G., & Paragliola, G. (2020) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2020.101964 · Cited by 320 · see citing works
Task Co-Offloading for D2D-Assisted Mobile Edge Computing in Industrial Internet of Things
Dai, X., Xiao, Z., Jiang, H., Alazab, M., Lui, J. C. S., Dustdar, S., & Liu, J. (2023) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2022.3158974 · Cited by 304 · see citing works
Secure Transmission Scheme Based on Joint Radar and Communication in Mobile Vehicular Networks
Yao, Y., Shu, F., Li, Z., Cheng, X., & Wu, L. (2023) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2023.3271452 · Cited by 278 · see citing works
Deconstructing the human algorithms for exploration
Gershman, S. J. (2018) · Cognition · doi:10.1016/j.cognition.2017.12.014 · Cited by 267 · see citing works
Reinforcement learning for iot security: A comprehensive survey
Uprety, A., & Rawat, D. B. (2021) · IEEE Internet of Things Journal · doi:10.1109/jiot.2020.3040957 · Cited by 214 · see citing works
Fairness in Learning: Classic and Contextual Bandits
Joseph, M., Kearns, M., Morgenstern, J., & Roth, A. (2016) · arXiv · doi:10.48550/arxiv.1605.07139 · Cited by 199 · see citing works
“Without a fairness constraint, it is known that it is possible to guarantee non-trivial regret to the optimal policy after only T = O ( k ) many rounds [Auer et al., 2002].”
BackgroundMethodInfluentialDynamic Online Pricing with Incomplete Information Using Multiarmed Bandit Experiments
Misra, K., Schwartz, E. M., & Abernethy, J. (2019) · Marketing Science · doi:10.1287/mksc.2018.1129 · Cited by 183 · see citing works
Interactive collaborative filtering
Zhao, X., Zhang, W., & Wang, J. (2013) · Proceedings of the 22nd ACM international conference on Information & Knowledge Management · doi:10.1145/2505515.2505690 · Cited by 165 · see citing works
Degenerate Feedback Loops in Recommender Systems
Jiang, R., Chiappa, S., Lattimore, T., György, A., & Kohli, P. (2019) · Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3306618.3314288 · Cited by 232 · see citing works
Learning Based Content Caching and Sharing for Wireless Networks
Song, J., Sheng, M., Quek, T. Q. S., Xu, C., & Wang, X. (2017) · IEEE Transactions on Communications · doi:10.1109/tcomm.2017.2713384 · Cited by 144 · see citing works
PopTherapy: Coping with Stress through Pop-Culture
Paredes, P., Giald-Bachrach, R., Czerwinski, M., Roseway, A., Rowan, K., & Hernandez, J. (2014) · Proceedings of the 8th International Conference on Pervasive Computing Technologies for Healthcare · doi:10.4108/icst.pervasivehealth.2014.255070 · Cited by 152 · see citing works
Learning State Selection for Reconfigurable Antennas: A Multi-Armed Bandit Approach
Gulati, N., & Dandekar, K. R. (2014) · IEEE Transactions on Antennas and Propagation · doi:10.1109/tap.2013.2276414 · Cited by 69 · see citing works
BACombo—Bandwidth-Aware Decentralized Federated Learning
Jiang, J., Hu, L., Hu, C., Liu, J., & Wang, Z. (2020) · Electronics · doi:10.3390/electronics9030440 · Cited by 47 · see citing works
Bayesian optimization for maximum power point tracking in photovoltaic power plants
Abdelrahman, H., Berkenkamp, F., Poland, J., & Krause, A. (2016) · 2016 European Control Conference (ECC) · doi:10.1109/ecc.2016.7810598 · Cited by 45 · see citing works
Algorithmic and Ethical Aspects of Recommender Systems in e-Commerce
Paraschakis, D. (2018) · doi:10.24834/2043/24268 · Cited by 35 · see citing works
Forman, E. M., Berry, M. P., Butryn, M. L., Hagerman, C. J., Huang, Z., Juarascio, A. S., LaFata, E. M., Ontañón, S., Tilford, J. M., & Zhang, F. (2023) · Contemporary Clinical Trials · doi:10.1016/j.cct.2022.107029 · Cited by 24 · see citing works
Automated game balancing of asymmetric video games
Beau, P., & Bakkes, S. (2016) · 2016 IEEE Conference on Computational Intelligence and Games (CIG) · doi:10.1109/cig.2016.7860432 · Cited by 22 · see citing works
Interpretable Goal Recognition in the Presence of Occluded Factors for Autonomous Vehicles
Hanna, J. P., Rahman, A., Fosong, E., Eiras, F., Dobre, M., Redford, J., Ramamoorthy, S., & Albrecht, S. V. (2021) · 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros51168.2021.9635903 · Cited by 21 · see citing works
Adaptive Methods in Machine Learning
Clerc, M., Daucé, E., & Mattout, J. (2016) · Brain–Computer Interfaces 1 · doi:10.1002/9781119144977.ch10 · Cited by 12 · see citing works
Interpretable Goal-based Prediction and Planning for Autonomous Driving
Albrecht, S. V., Brewitt, C., Wilhelm, J., Gyevnar, B., Eiras, F., Dobre, M., & Ramamoorthy, S. (2021) · 2021 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra48506.2021.9560849 · Cited by 12 · see citing works
A Sleeping, Recovering Bandit Algorithm for Optimizing Recurring Notifications
Yancey, K. P., & Settles, B. (2020) · Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining · doi:10.1145/3394486.3403351 · Cited by 5 · see citing works
Language Agents as Optimizable Graphs
Zhuge, M., Wang, W., Kirsch, L., Faccio, F., Khizbullin, D., & Schmidhuber, J. (2024) · arXiv · doi:10.48550/arxiv.2402.16823 · Cited by 3 · see citing works
DarwinX: Evolving Agent Harnesses Through Natural Selection
Zhang, Y., Dai, Y., Tan, J., Yang, L., Mullur, R., Hoang, T., Hu, Z., Zhu, J., Mui, P., Savarese, S., et al. (2026) · arXiv · doi:10.48550/arxiv.2608.07545 · Cited by 1 · see citing works
Aubert, C., Raineau, Y., & Raynal, M. (2026) · European Review of Agricultural Economics · doi:10.1093/erae/jbaf065
On The Statistical Limits of Self-Improving Agents
Wang, C. L., Dorchen, K., & Jin, P. (2025) · arXiv · doi:10.48550/arxiv.2510.04399
ShinkaEvolve: Towards Open-Ended And Sample-Efficient Program Evolution
Lange, R. T., Imajuku, Y., & Cetin, E. (2025) · arXiv · doi:10.48550/arxiv.2509.19349
Zhang, J., Zhao, B., Yang, W., Foerster, J., Clune, J., Jiang, M., Devlin, S., & Shavrina, T. (2026) · arXiv · doi:10.48550/arxiv.2603.19461
Algorithmic monoculture and social welfare
(2023) · The Societal Impacts of Algorithmic Decision-Making · doi:10.1145/3603195.3603202
DORA Explorer: Improving the Exploration Ability of LLMs Without Training
Gurjar, P., Ishmam, M. F., & Marino, K. (2026) · arXiv · doi:10.48550/arxiv.2604.17244
“DORA achieves UCB-competitive performance on MAB and consistent gains across the TALES environments.”
Influential
- Authors
- Auer, P., Cesa-Bianchi, N., & Fischer, P.
- Affiliations
- Graz University of Technology (AT)
- TU Dortmund University (DE)
- Venue
- Machine Learning
- Published
- 2002-05
- Volume
- 47
- Issue
- 2-3
- Pages
- 235-256
- ISSN
- 0885-6125 (ISSN-L)
- 1573-0565
- OpenAlex
- W2168405694
- Cited by
- 6,033
- References
- 1
- Citations by year
- 2026: 343 · 2025: 346 · 2024: 353 · 2023: 361 · 2022: 405 · 2021: 559 · 2020: 603 · 2019: 478 · 2018: 417 · 2017: 345 · 2016: 325 · 2015: 306 · 2014: 291 · 2013: 234 · 2012: 209
- FWCI
- 6.24
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Licence
- Locations
- Machine Learning (publishedVersion)
- Technical University of Denmark, DTU Orbit (Technical University of Denmark, DTU) (submittedVersion)
- Topics
- Social Sciences › Decision Sciences › Management Science and Operations Research › Advanced Bandit Algorithms Research
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Computer Science › Computer Networks and Communications › Optimization and Search Problems
- Keywords
- multi-armed bandits, exploration-exploitation balance, logarithmic regret, reinforcement learning policy
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
