The DOI matches its Crossref record; 51 of the 53 references it cites were found in a registry or an index.
Modeling Medical Treatment Using Markov Decision Processes
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Currently, most treatment decisions in the medical domain are made via ad-hoc or heuristic approaches, but there is a growing body of evidence that such complex treatment decisions are better handled through modeling rather than intuition alone [18,21].”
From Artificial intelligence framework for simulating clinical decision-making: A Markov decision process approach (2013) · cites this paper as [18]
In the citing paper
“Markov decision processes (MDPs) are one efficient technique for determining such optimal sequential decisions (termed a 'policy') in dynamic and uncertain environments [18,19], and have been explored in medical decision-making problems in recent years [18,20].”
From Artificial intelligence framework for simulating clinical decision-making: A Markov decision process approach (2013) · cites this paper as [18]
In the citing paper
“As noted elsewhere, the collection of such 'natural history' disease data is fraught with many challenges, ethical and otherwise, particularly if such untreated conditions pose significant health risks [18].”
From Artificial intelligence framework for simulating clinical decision-making: A Markov decision process approach (2013) · cites this paper as [18]
In the citing paper
“However, despite such applicability of sequential decisionmaking techniques like MDPs to medical decision-making, there have been relatively few applications in healthcare [18,19].”
From Artificial intelligence framework for simulating clinical decision-making: A Markov decision process approach (2013) · cites this paper as [18]
In the citing paper
“The work presented here adds to a growing body of evidence that such complex treatment decisions may be better handled through modeling than intuition alone [18,21].”
From Artificial intelligence framework for simulating clinical decision-making: A Markov decision process approach (2013) · cites this paper as [18]
In the citing paper
“MDPs (and their partially observable cousins) directly address many of the challenges faced in clinical decision-making [17,18].”
From Artificial intelligence framework for simulating clinical decision-making: A Markov decision process approach (2013) · cites this paper as [18]
Citations
Cites
Tversky, A., & Kahneman, D. (1974) · Science · doi:10.1126/science.185.4157.1124 · Cited by 28,810
Judgment under Uncertainty: Heuristics and Biases
Tversky, A., & Kahneman, D. (1975) · Utility, Probability, and Human Decision Making · doi:10.1007/978-94-010-1834-0_8 · Cited by 22,650
Bellman, R. E. (2010) · doi:10.1515/9781400835386 · Cited by 14,588
Methods for the Economic Evaluation of Health Care Programmes
Drummond, M. E., Sculpher, M. J., Torrance, G. W., O’Brien, B. J., & Stoddart, G. L. (2005) · doi:10.1093/oso/9780198529446.001.0001 · Cited by 13,648
(1995) · Cited by 10,558
Availability: A heuristic for judging frequency and probability
Tversky, A., & Kahneman, D. (1973) · Cognitive Psychology · doi:10.1016/0010-0285(73)90033-9 · Cited by 9,984 · cited 1×
“However, there is a growing feeling among medical practitioners that some treatment decisions are too complicated to solve accurately using intuition alone [1, 2].”
BackgroundMarkov decision processes: Discrete stochastic dynamic programming
Hazeghi, K., & Puterman, M. L. (1995) · Journal of the American Statistical Association · doi:10.2307/2291177 · Cited by 8,037
Cost-Effectiveness in Health and Medicine
(1996) · doi:10.1093/oso/9780195108248.001.0001 · Cited by 7,193
Time Discounting and Time Preference: A Critical Review
Frederick, S., Loewenstein, G., & O’Donoghue, T. (2002) · Journal of Economic Literature · doi:10.1257/002205102320161311 · Cited by 6,106
Puterman, M. L. (1994) · Wiley series in probability and statistics · doi:10.1002/9780470316887 · Cited by 6,057
Research works printed but not linked: 1
- Roberts, M.S. and D.C. Angus (2002). The optimal timing of liver transplantation: Final report R01 HS09694. University of Pittsburgh, Pittsburgh, PA.
Cited by
Sorted by citations
Komorowski, M., Celi, L. A., Badawi, O., Gordon, A. C., & Faisal, A. A. (2018) · Nature Medicine · doi:10.1038/s41591-018-0213-5 · Cited by 1,290
Reinforcement Learning in Healthcare: A Survey
Yu, C., Liu, J., Nemati, S., & Yin, G. (2021) · ACM Computing Surveys · doi:10.1145/3477600 · Cited by 783
Bennett, C. C., & Hauser, K. (2013) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2012.12.003 · Cited by 378
“Markov decision processes (MDPs) are one efficient technique for determining such optimal sequential decisions (termed a “policy”) in dynamic and uncertain environments [18,19], and have been explored in medical decision-making problems in recent years [18,20].”
InfluentialMarkov decision processes: a tool for sequential decision making under uncertainty
Alagoz, O., Hsu, H., Schaefer, A. J., & Roberts, M. S. (2010) · Medical Decision Making · doi:10.1177/0272989x09353194 · Cited by 266
Self-citationData, M. C. (2020) · OAPEN (The OAPEN Foundation) · doi:10.1007/978-3-319-43742-2 · Cited by 273
Metadata
- Authors
- Schaefer, A. J., Bailey, M. D., Shechter, S. M., & Roberts, M. S.
- Affiliations
- University of Pittsburgh (US)
- Venue
- International Series in Operations Research & Management Science
- Publisher
- Springer US
- Published
- 2005
- Pages
- 593-612
- ISSN
- 0884-8289 (ISSN-L)
- OpenAlex
- W58990062
- Cited by
- 155
- References
- 53
- Citations by year
- 2026: 3 · 2025: 7 · 2024: 11 · 2023: 13 · 2022: 6 · 2021: 13 · 2020: 8 · 2019: 17 · 2018: 15 · 2017: 9 · 2016: 5 · 2015: 12 · 2014: 3 · 2013: 6 · 2012: 5
- FWCI
- 8.16
- Type
- book chapter
- Language
- en
- Open access
- closed
- Licence
- Locations
- International series in management science/operations research/International series in operations research & management science (publishedVersion)
- Funders
- National Science Foundation
- Topics
- Social Sciences › Economics, Econometrics and Finance › Economics and Econometrics › Health Systems, Economic Evaluations, Quality of Life
- Social Sciences › Decision Sciences › Management Science and Operations Research › Complex Systems and Decision Making
- Physical Sciences › Engineering › Control and Systems Engineering › Adaptive Dynamic Programming Control
- Keywords
- Markov decision process, treatment decision-making, sequential decision making, stochastic dynamic programming, treatment planning, clinical decision support
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
