Partially Observed Markov Decision Processes: From Filtering to Controlled Sensing
The DOI matches its Crossref record; 16 of the 16 references it cites were found in a registry or an index.
Abstract
Covering formulation, algorithms, and structural results, and linking theory to real-world applications in controlled sensing (including social learning, adaptive radars and sequential detection), this book focuses on the conceptual foundations of partially observed Markov decision processes (POMDPs). It emphasizes structural results in stochastic dynamic programming, enabling graduate students and researchers in engineering, operations research, and economics to understand the underlying unifying themes without getting weighed down by mathematical technicalities. Bringing together research from across the literature, the book provides an introduction to nonlinear filtering followed by a systematic development of stochastic dynamic programming, lattice programming and reinforcement learning for POMDPs. Questions addressed in the book include: when does a POMDP have a threshold optimal policy? When are myopic policies optimal? How do local and global decision makers interact in adaptive decision making in multi-agent social learning where there is herding and data incest? And how can sophisticated radars and sensors adapt their sensing in real time?
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Boyd, S., & Vandenberghe, L. (2004) · doi:10.1017/cbo9780511804441 · Cited by 56,601
A New Approach to Linear Filtering and Prediction Problems
Kalman, R. E. (1960) · Journal of Basic Engineering · doi:10.1115/1.3662552 · Cited by 31,751
The Elements of Statistical Learning
Hastie, T., Friedman, J., & Tibshirani, R. (2001) · Springer Series in Statistics · doi:10.1007/978-0-387-21606-5 · Cited by 24,162
A tutorial on hidden Markov models and selected applications in speech recognition
Rabiner, L. R. (1989) · Proceedings of the IEEE · doi:10.1109/5.18626 · Cited by 22,743
Estimation with Applications to Tracking and Navigation: Theory, Algorithms and Software
Bar‐Shalom, Y., Li, X. ‐., & Kirubarajan, T. (2002) · doi:10.1002/0471221279 · Cited by 6,782
Finite-time Analysis of the Multiarmed Bandit Problem
Auer, P., Cesa-Bianchi, N., & Fischer, P. (2002) · Machine Learning · doi:10.1023/a:1013689704352 · Cited by 6,033
Good News and Bad News: Representation Theorems and Applications
Milgrom, P. R. (1981) · The Bell Journal of Economics · doi:10.2307/3003562 · Cited by 3,123
Acceleration of Stochastic Approximation by Averaging
Polyak, B. T., & Juditsky, A. B. (1992) · SIAM Journal on Control and Optimization · doi:10.1137/0330046 · Cited by 1,827
(2000) · doi:10.2460/ajvr.67.2.323 · Cited by 1,621
The Complexity of Markov Decision Processes
Papadimitriou, C. H., & Tsitsiklis, J. N. (1987) · Mathematics of Operations Research · doi:10.1287/moor.12.3.441 · Cited by 1,329
Cited by
Sublinear regret for learning POMDPs
Xiong, Y., Chen, N., Gao, X., & Zhou, X. (2022) · Production and Operations Management · doi:10.1111/poms.13778 · Cited by 6
Metadata
- Authors
- Krishnamurthy, V.
- Venue
- Cambridge University Press eBooks
- Publisher
- Cambridge University Press
- Published
- 2016-03-21
- OpenAlex
- W4239466562
- Cited by
- 326
- References
- 16
- FWCI
- 13.66
- Type
- book
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Partially Observed Markov Decision Processes: From Filtering to Controlled Sensing. https://app.cochat.ai/papers/10.1017/cbo9781316471104
- The paper
- Krishnamurthy, V. (2016). Partially Observed Markov Decision Processes: From Filtering to Controlled Sensing. Cambridge University Press eBooks. https://doi.org/10.1017/cbo9781316471104
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.