The DOI matches its Crossref record; 25 of the 25 references it cites were found in a registry or an index.
Simulation-based optimization of Markov reward processes
Open accessAbstract
This paper proposes a simulation-based algorithm for optimizing the average reward in a finite-state Markov reward process that depends on a set of parameters. As a special case, the method applies to Markov decision processes where optimization takes place within a parametrized set of policies. The algorithm relies on the regenerative structure of finite-state Markov processes, involves the simulation of a single sample path, and can be implemented online. A convergence result (with probability 1) is provided.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225
Reinforcement Learning: An Introduction
(2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 24,496
(1995) · Cited by 10,558
Markov decision processes: Discrete stochastic dynamic programming
Hazeghi, K., & Puterman, M. L. (1995) · Journal of the American Statistical Association · doi:10.2307/2291177 · Cited by 8,037
Markov Decision Processes: Discrete Stochastic Dynamic Programming
Puterman, M. L. (1994) · Wiley Series in Probability and Statistics · doi:10.1002/9780470316887 · Cited by 6,024
Watkins, C. (1989) · OpenGrey (Institut de l'Information Scientifique et Technique) · doi:10.17863/cam.132164 · Cited by 5,297
(1999) · Cited by 5,147
Wiley Series in Probability and Statistics
(2008) · Wiley series in probability and statistics · doi:10.1002/series1345 · Cited by 4,217
(2012) · Encyclopedia of the Sciences of Learning · doi:10.1007/978-1-4419-1428-6_5086 · Cited by 3,515
Neuronlike adaptive elements that can solve difficult learning control problems
Barto, A. G., Sutton, R. S., & Anderson, C. W. (1983) · IEEE Transactions on Systems, Man, and Cybernetics · doi:10.1109/tsmc.1983.6313077 · Cited by 3,233
Cited by
Konda, V. R., & Tsitsiklis, J. N. (2003) · SIAM Journal on Control and Optimization · doi:10.1137/s0363012901385691 · Cited by 709
Self-citationNatural actor–critic algorithms
Bhatnagar, S., Sutton, R. S., Ghavamzadeh, M., & Lee, M. (2009) · Automatica · doi:10.1016/j.automatica.2009.07.008 · Cited by 559
Learning to trade via direct reinforcement
Moody, J., & Saffell, M. (2001) · IEEE Transactions on Neural Networks · doi:10.1109/72.935097 · Cited by 535
Dynamic Programming and Suboptimal Control: A Survey from ADP to MPC*
Bertsekas, D. P. (2005) · European Journal of Control · doi:10.3166/ejc.11.310-334 · Cited by 366
Reinforcement Learning Through Modulation of Spike-Timing-Dependent Synaptic Plasticity
Florian, R. V. (2007) · Neural Computation · doi:10.1162/neco.2007.19.6.1468 · Cited by 358
Congestion-dependent pricing of network services
Paschalidis, I. C., & Tsitsiklis, J. N. (2000) · IEEE/ACM Transactions on Networking · doi:10.1109/90.842140 · Cited by 352
Self-citationVariance Reduction Techniques for Gradient Estimates in Reinforcement Learning
Greensmith, E., Bartlett, P. L., & Baxter, J. (2004) · doi:10.5555/1005332.1044710 · Cited by 300
Minimizing fuel emissions by optimizing vessel schedules in liner shipping with uncertain port times
Qi, X., & Song, D.-P. (2012) · Transportation Research Part E: Logistics and Transportation Review · doi:10.1016/j.tre.2012.02.001 · Cited by 284
Giuliani, M., Castelletti, A., Pianosi, F., Mason, E., & Reed, P. M. (2016) · Journal of Water Resources Planning and Management · doi:10.1061/(asce)wr.1943-5452.0000570 · Cited by 257
Metadata
- Authors
- Marbach, P., & Tsitsiklis, J. N.
- Venue
- IEEE Transactions on Automatic Control
- Published
- 2001
- Volume
- 46
- Issue
- 2
- Pages
- 191-209
- ISSN
- 0018-9286 (ISSN-L)
- OpenAlex
- W2120465407
- Cited by
- 306
- References
- 25
- FWCI
- 28.79
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.