The DOI matches its Crossref record.
Bandit Problems, Sequential Allocation of Experiments.
Open accessAbstract
Our purpose in writing this monograph is to give a comprehensive treatment of the subject. We define bandit problems and give the necessary foundations in Chapter 2. Many of the important results that have appeared in the literature are presented in later chapters; these are interspersed with new results. We give proofs unless they are very easy or the result is not used in the sequel. We have simplified a number of arguments so many of the proofs given tend to be conceptual rather than calculational. All results given have been incorporated into our style and notation. The exposition is aimed at a variety of types of readers. Bandit problems and the associated mathematical and technical issues are developed from first principles. Since we have tried to be comprehens ive the mathematical level is sometimes advanced; for example, we use measure-theoretic notions freely in Chapter 2. But the mathema tically uninitiated reader can easily sidestep such discussion when it occurs in Chapter 2 and elsewhere. We have tried to appeal to graduate students and professionals in engineering, biometry, econ omics, management science, and operations research, as well as those in mathematics and statistics. The monograph could serve as a reference for professionals or as a telA in a semester or year-long graduate level course.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cited by
Reinforcement Learning: A Survey
Kaelbling, L. P., Littman, M. L., & Moore, A. W. (1996) · Journal of Artificial Intelligence Research · doi:10.1613/jair.301 · Cited by 8,972
A contextual-bandit approach to personalized news article recommendation
Li, L., Chu, W., Langford, J., & Schapire, R. E. (2010) · Proceedings of the 19th international conference on World wide web · doi:10.1145/1772690.1772758 · Cited by 2,562
Bayesian Experimental Design: A Review
Chaloner, K., & Verdinelli, I. (1995) · Statistical Science · doi:10.1214/ss/1177009939 · Cited by 1,839
Cohen, J. D., McClure, S. M., & Yu, A. J. (2007) · Philosophical Transactions of the Royal Society B Biological Sciences · doi:10.1098/rstb.2007.2098 · Cited by 1,121
Prioritized sweeping: Reinforcement learning with less data and less time
Moore, A. W., & Atkeson, C. G. (1993) · Machine Learning · doi:10.1007/bf00993104 · Cited by 823
Introduction to Multi-Armed Bandits
Slivkins, A. (2019) · Foundations and Trends® in Machine Learning · doi:10.1561/2200000068 · Cited by 780
An experimental design for the development of adaptive treatment strategies
Murphy, S. A. (2005) · Statistics in Medicine · doi:10.1002/sim.2022 · Cited by 746
Bala, V., & Goyal, S. (1998) · Review of Economic Studies · doi:10.1111/1467-937x.00059 · Cited by 746
Robust Identities or Nonentities? Typecasting in the Feature‐Film Labor Market
Zuckerman, E. W., Kim, T., Ukanwa, K., & von Rittmann, J. (2003) · American Journal of Sociology · doi:10.1086/377518 · Cited by 590
Continuously adaptive continuous queries over streams
Madden, S., Shah, M., Hellerstein, J. M., & Raman, V. (2002) · Proceedings of the 2002 ACM SIGMOD international conference on Management of data · doi:10.1145/564691.564698 · Cited by 583
(2008) · ScholarlyCommons (University of Pennsylvania) · Cited by 581
Decision theory, reinforcement learning, and the brain
Dayan, P., & Daw, N. D. (2008) · Cognitive, Affective, & Behavioral Neuroscience · doi:10.3758/cabn.8.4.429 · Cited by 580
(2021) · doi:10.1109/tse.2019.2946563 · Cited by 574
Reinforcement learning is direct adaptive optimal control
(1992) · IEEE Control Systems · doi:10.1109/37.126844 · Cited by 524
Learning by Doing and the Choice of Technology
Jovanovic, B., & Nyarko, Y. (1996) · Econometrica · doi:10.2307/2171832 · Cited by 481
On Adaptation, Maximization, and Reinforcement Learning Among Cognitive Strategies.
Erev, I., & Barron, G. (2005) · Psychological Review · doi:10.1037/0033-295x.112.4.912 · Cited by 475
The Epistemic Benefit of Transient Diversity
Zollman, K. J. S. (2010) · Erkenntnis · doi:10.1007/s10670-009-9194-6 · Cited by 460
Unbiased offline evaluation of contextual-bandit-based news article recommendation algorithms
Li, L., Chu, W., Langford, J., & Wang, X. (2011) · Proceedings of the fourth ACM international conference on Web search and data mining · doi:10.1145/1935826.1935878 · Cited by 443
Linearly Parameterized Bandits
Rusmevichientong, P., & Tsitsiklis, J. N. (2010) · Mathematics of Operations Research · doi:10.1287/moor.1100.0446 · Cited by 427
A modern Bayesian look at the multi‐armed bandit
Scott, S. L. (2010) · Applied Stochastic Models in Business and Industry · doi:10.1002/asmb.874 · Cited by 406
Chasing a Moving Target: Exploitation and Exploration in Dynamic Environments
Posen, H. E., & Levinthal, D. A. (2012) · Management Science · doi:10.1287/mnsc.1110.1420 · Cited by 380
Reinforcement learning improves behaviour from evaluative feedback
Littman, M. L. (2015) · Nature · doi:10.1038/nature14540 · Cited by 368
Cognitive Medium Access: Exploration, Exploitation, and Competition
Lai, L., El Gamal, H., Jiang, H., & Poor, H. V. (2011) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2010.65 · Cited by 351
A re‐examination of probability matching and rational choice
Shanks, D. R., Tunney, R. J., & McCarthy, J. D. (2002) · Journal of Behavioral Decision Making · doi:10.1002/bdm.413 · Cited by 322
Sequential design of computer experiments for the estimation of a probability of failure
Bect, J., Ginsbourger, D., Li, L., Picheny, V., & Vazquez, E. (2012) · Statistics and Computing · doi:10.1007/s11222-011-9241-4 · Cited by 321
The Communication Structure of Epistemic Communities
Zollman, K. J. S. (2007) · Philosophy of Science · doi:10.1086/525605 · Cited by 316
Bee foraging in uncertain environments using predictive hebbian learning
Montague, P. R., Dayan, P., Person, C., & Sejnowski, T. J. (1995) · Nature · doi:10.1038/377725a0 · Cited by 311
Dynamic Assortment with Demand Learning for Seasonal Consumer Goods
Caro, F., & Gallien, J. (2007) · Management Science · doi:10.1287/mnsc.1060.0613 · Cited by 306
Empirical evaluation methods for multiobjective reinforcement learning algorithms
Vamplew, P., Dazeley, R., Berry, A., Issabekov, R., & Dekker, E. (2011) · Machine Learning · doi:10.1007/s10994-010-5232-5 · Cited by 275
An object-oriented representation for efficient reinforcement learning
Diuk, C., Cohen, A., & Littman, M. L. (2008) · Proceedings of the 25th international conference on Machine learning - ICML '08 · doi:10.1145/1390156.1390187 · Cited by 226
Metadata
- Authors
- Jones, P. W., Berry, D. A., & Fristedt, B.
- Venue
- The Journal of the Operational Research Society
- Publisher
- JSTOR
- Published
- 1987-08
- Volume
- 38
- Issue
- 8
- Pages
- 773
- ISSN
- 0160-5682 (ISSN-L)
- OpenAlex
- W2058879737
- Cited by
- 566
- References
- 0
- FWCI
- 11.87
- Type
- journal article
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.