Cited by 155Health Systems, Economic Evaluations, Quality of LifeComplex Systems and Decision MakingAdaptive Dynamic Programming Control

The DOI matches its Crossref record; 51 of the 53 references it cites were found in a registry or an index.

Modeling Medical Treatment Using Markov Decision Processes

Schaefer, Andrew J., Bailey, Matthew D., Shechter, Steven M., Roberts, Mark S.·2005·International Series in Operations Research & Management Science

DOI 10.1007/1-4020-8066-2_23

Open this paper in CoChat — free account

Integrity signals

Registry

Crossref · 10.1007/1-4020-8066-2_23

Cites

51 of 51 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

We couldn’t retrieve this paper’s full text, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Judgment under Uncertainty: Heuristics and Biases: Biases in judgments reveal some heuristics of thinking under uncertainty.
Tversky, A., & Kahneman, D. (1974) · Science · doi:10.1126/science.185.4157.1124 · Cited by 28,810
Judgment under Uncertainty: Heuristics and Biases
Tversky, A., & Kahneman, D. (1975) · Utility, Probability, and Human Decision Making · doi:10.1007/978-94-010-1834-0_8 · Cited by 22,650
Dynamic Programming
Bellman, R. E. (2010) · doi:10.1515/9781400835386 · Cited by 14,588
Methods for the Economic Evaluation of Health Care Programmes
Drummond, M. E., Sculpher, M. J., Torrance, G. W., O’Brien, B. J., & Stoddart, G. L. (2005) · doi:10.1093/oso/9780198529446.001.0001 · Cited by 13,648
Untitled work
(1995) · Cited by 10,558
Availability: A heuristic for judging frequency and probability
Tversky, A., & Kahneman, D. (1973) · Cognitive Psychology · doi:10.1016/0010-0285(73)90033-9 · Cited by 9,984 · cited 1×
“However, there is a growing feeling among medical practitioners that some treatment decisions are too complicated to solve accurately using intuition alone [1, 2].”
Background
Markov decision processes: Discrete stochastic dynamic programming
Hazeghi, K., & Puterman, M. L. (1995) · Journal of the American Statistical Association · doi:10.2307/2291177 · Cited by 8,037
Cost-Effectiveness in Health and Medicine
(1996) · doi:10.1093/oso/9780195108248.001.0001 · Cited by 7,193
Time Discounting and Time Preference: A Critical Review
Frederick, S., Loewenstein, G., & O’Donoghue, T. (2002) · Journal of Economic Literature · doi:10.1257/002205102320161311 · Cited by 6,106
Markov Decision Processes
Puterman, M. L. (1994) · Wiley series in probability and statistics · doi:10.1002/9780470316887 · Cited by 6,057

Show all 53 references

Research works printed but not linked: 1

Roberts, M.S. and D.C. Angus (2002). The optimal timing of liver transplantation: Final report R01 HS09694. University of Pittsburgh, Pittsburgh, PA.

Sorted by citations

The artificial intelligence clinician learns optimal treatment strategies for sepsis in intensive care
Komorowski, M., Celi, L. A., Badawi, O., Gordon, A. C., & Faisal, A. A. (2018) · Nature Medicine · doi:10.1038/s41591-018-0213-5 · Cited by 1,290
Reinforcement Learning in Healthcare: A Survey
Yu, C., Liu, J., Nemati, S., & Yin, G. (2021) · ACM Computing Surveys · doi:10.1145/3477600 · Cited by 783
Artificial intelligence framework for simulating clinical decision-making: A Markov decision process approach
Bennett, C. C., & Hauser, K. (2013) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2012.12.003 · Cited by 378
“Markov decision processes (MDPs) are one efficient technique for determining such optimal sequential decisions (termed a “policy”) in dynamic and uncertain environments [18,19], and have been explored in medical decision-making problems in recent years [18,20].”
Influential
Markov decision processes: a tool for sequential decision making under uncertainty
Alagoz, O., Hsu, H., Schaefer, A. J., & Roberts, M. S. (2010) · Medical Decision Making · doi:10.1177/0272989x09353194 · Cited by 266
Self-citation
1001985.pdf
Data, M. C. (2020) · OAPEN (The OAPEN Foundation) · doi:10.1007/978-3-319-43742-2 · Cited by 273
Artificial Intelligence in Pathology
Chang, H. Y., Jung, C. K., Woo, J. I., Lee, S.-H., Cho, J., Kim, S. W., & Kwak, T.-Y. (2018) · Journal of Pathology and Translational Medicine · doi:10.4132/jptm.2018.12.16 · Cited by 256
Confronting Deep Uncertainties in Risk Analysis
Cox, L. A. (2012) · Risk Analysis · doi:10.1111/j.1539-6924.2012.01792.x · Cited by 211
OR Forum—A POMDP Approach to Personalize Mammography Screening Decisions
Ayer, T., Alagöz, O., & Stout, N. K. (2012) · Operations Research · doi:10.1287/opre.1110.1019 · Cited by 199
The Optimal Time to Initiate HIV Therapy Under Ordered Health States
Shechter, S. M., Bailey, M. D., Schaefer, A. J., & Roberts, M. S. (2008) · Operations Research · doi:10.1287/opre.1070.0480 · Cited by 148
Self-citation
Literature review on multi-appointment scheduling problems in hospitals
Marynissen, J., & Demeulemeester, E. L. (2018) · European Journal of Operational Research · doi:10.1016/j.ejor.2018.03.001 · Cited by 145
Health Big Data Analytics: A Technology Survey
Harerimana, G., Jang, B., Kim, J. W., & Park, H. K. (2018) · IEEE Access · doi:10.1109/access.2018.2878254 · Cited by 119
The value of mHealth for managing chronic conditions
Agnihothri, S., Cui, L., Delasay, M., & Rajan, B. (2020) · Health Care Management Science · doi:10.1007/s10729-018-9458-2 · Cited by 96
Optimizing the Start Time of Statin Therapy for Patients with Diabetes
Denton, B. T., Kurt, M., Shah, N. D., Bryant, S. C., & Smith, S. A. (2009) · Medical Decision Making · doi:10.1177/0272989x08329462 · Cited by 88
Robust Markov Decision Processes: Beyond Rectangularity
Goyal, V., & Grand-Clément, J. (2022) · Mathematics of Operations Research · doi:10.1287/moor.2022.1259 · Cited by 82
Analyzing Patient Trajectories With Artificial Intelligence
Allam, A., Feuerriegel, S., Rebhan, M., & Krauthammer, M. (2021) · Journal of Medical Internet Research · doi:10.2196/29812 · Cited by 77
Reinforcement learning strategies in cancer chemotherapy treatments: A review
Yang, C., Shiranthika, C., Wang, C.-Y., Chen, K.-W., & Sumathipala, S. (2022) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2022.107280 · Cited by 74
Markov Decision Processes for Screening and Treatment of Chronic Diseases
Steimle, L. N., & Denton, B. T. (2017) · International series in management science/operations research/International series in operations research & management science · doi:10.1007/978-3-319-47766-4_6 · Cited by 74
Individualization of pharmacological anemia management using reinforcement learning
Gawęda, A. E., Muezzinoglu, M. K., Aronoff, G. R., Jacobs, A. A., Żurada, J. M., & Brier, M. E. (2005) · Neural Networks · doi:10.1016/j.neunet.2005.06.020 · Cited by 72
Integrated Anesthesiologist and Room Scheduling for Surgeries: Methodology and Application
Rath, S., Rajaram, K., & Mahajan, A. (2017) · Operations Research · doi:10.1287/opre.2017.1634 · Cited by 77
Elective Patient Admission and Scheduling under Multiple Resource Constraints
Barz, C., & Rajaram, K. (2015) · Production and Operations Management · doi:10.1111/poms.12395 · Cited by 69
Markov Models and Cost Effectiveness Analysis: Applications in Medical Research
Komorowski, M., & Raffa, J. D. (2016) · Springer eBooks · doi:10.1007/978-3-319-43742-2_24 · Cited by 67
Generalized Markov models of infectious disease spread: A novel framework for developing dynamic health policies
Yaesoubi, R., & Cohen, T. (2011) · European Journal of Operational Research · doi:10.1016/j.ejor.2011.07.016 · Cited by 66
The Effect of Budgetary Restrictions on Breast Cancer Diagnostic Decisions
Ayvaci, M., Alagöz, O., & Burnside, E. S. (2012) · Manufacturing & Service Operations Management · doi:10.1287/msom.1110.0371 · Cited by 59
Personalized screening intervals for biomarkers using joint models for longitudinal and survival data
Rizopoulos, D., Taylor, J. M. G., van Rosmalen, J., Steyerberg, E. W., & Takkenberg, J. J. M. (2015) · Biostatistics · doi:10.1093/biostatistics/kxv031 · Cited by 52
Energy Efficient Computation Offloading Mechanism in Multi-Server Mobile Edge Computing—An Integer Linear Optimization Approach
Khan, P. W., Abbas, K., Shaiba, H. A., Muthanna, A., Abuarqoub, A., & Khayyat, M. M. (2020) · Electronics · doi:10.3390/electronics9061010 · Cited by 50
The use of computer-interpretable clinical guidelines to manage care complexities of patients with multimorbid conditions: A review
Bilici, E., Despotou, G., & Arvanitis, T. N. (2018) · Digital Health · doi:10.1177/2055207618804927 · Cited by 44
Transatlantic transferability of a new reinforcement learning model for optimizing haemodynamic treatment for critically ill patients with sepsis
Roggeveen, L., el Hassouni, A., Ahrendt, J., Guo, T., Fleuren, L., Thoral, P., Girbes, A. R., Hoogendoorn, M., & Elbers, P. W. (2021) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2020.102003 · Cited by 41
Reinforcement learning-based expanded personalized diabetes treatment recommendation using South Korean electronic health records
Oh, S. H., Park, J., Lee, S. J., Kang, S. S., & Mo, J. (2022) · Expert Systems with Applications · doi:10.1016/j.eswa.2022.117932 · Cited by 41
From Data to Improved Decisions: Operations Research in Healthcare Delivery
Capan, M., Khojandi, A., Denton, B. T., Williams, K. D., Ayer, T., Chhatwal, J., Kurt, M., Mason, J. E., Roberts, M. S., Zaric, G. S., et al. (2017) · Medical Decision Making · doi:10.1177/0272989x17705636 · Cited by 37
Self-citation
Medical decision making: open research challenges
Denton, B. T., Alagöz, O., Holder, A. G., & Lee, E. K. (2011) · IIE Transactions on Healthcare Systems Engineering · doi:10.1080/19488300.2011.619157 · Cited by 31
Designing Personalized Treatment: An Application to Anticoagulation Therapy
Ibrahim, R., Küçükyazicı, B., Verter, V., Michel Gendreau, A., & Blostein, M. D. (2015) · Production and Operations Management · doi:10.1111/poms.12514 · Cited by 27
A proactive transfer policy for critical patient flow management
González, J., Ferrer, J., Cataldo, A., & Rojas, L. E. (2018) · Health Care Management Science · doi:10.1007/s10729-018-9437-7 · Cited by 27
Computational medication regimen for Parkinson’s disease using reinforcement learning
Kim, Y., Suescun, J., Schiess, M. C., & Jiang, X. (2021) · Scientific Reports · doi:10.1038/s41598-021-88619-4 · Cited by 30
How to Manage Red Alert in Emergency and Disaster Unit in the Hospital? Evidence From London
Heydari, M., Lai, K. K., & Xiaohu, Z. (2021) · Frontiers in Public Health · doi:10.3389/fpubh.2021.634417 · Cited by 23
Robustness of Proactive Intensive Care Unit Transfer Policies
Grand-Clément, J., Chan, C. W., Goyal, V., & Escobar, G. J. (2022) · Operations Research · doi:10.1287/opre.2022.2403 · Cited by 23
A dosing strategy model of deep deterministic policy gradient algorithm for sepsis patients
Lin, T., Zhang, X., Gong, J., Tan, R., Li, W., Wang, L., Pan, Y., Xu, X., & Gao, J. (2023) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-023-02175-7 · Cited by 22
Mining Major Transitions of Chronic Conditions in Patients with Multiple Chronic Conditions
Jaramillo, C. A., Faruqui, S. H. A., Pugh, M. J., & Alaeddini, A. (2017) · Methods of Information in Medicine · doi:10.3414/me16-01-0135 · Cited by 22
Optimization of Spirulina sp. cultivation using reinforcement learning with state prediction based on LSTM neural network
Doan, Y. T.-T., Ho, M.-T., Nguyen, H.-K., & Han, H.-D. (2021) · Journal of Applied Phycology · doi:10.1007/s10811-021-02488-y · Cited by 20
Online scheduling for outpatient services with heterogeneous patients and physicians
Su, H., Wan, G., & Wang, S. (2017) · Journal of Combinatorial Optimization · doi:10.1007/s10878-017-0216-z · Cited by 20
Optimization of Sequential Decision Making for Chronic Diseases: From Data to Decisions
Denton, B. T. (2018) · doi:10.1287/educ.2018.0184 · Cited by 20
Optimal treatment recommendations for diabetes patients using the Markov decision process along with the South Korean electronic health records
Oh, S. H., Lee, S. J., Noh, J., & Mo, J. (2021) · Scientific Reports · doi:10.1038/s41598-021-86419-4 · Cited by 20
Data-Driven Markov Decision Process Approximations for Personalized Hypertension Treatment Planning
Schell, G. J., Marrero, W. J., Lavieri, M. S., Sussman, J. B., & Hayward, R. A. (2016) · MDM Policy & Practice · doi:10.1177/2381468316674214 · Cited by 20
Stochastic dynamic resource allocation for HIV prevention and treatment: An approximate dynamic programming approach
Coşgun, Ö., & Büyüktahtakın, İ. E. (2018) · Computers & Industrial Engineering · doi:10.1016/j.cie.2018.01.018 · Cited by 19
A Markov decision process for modeling adverse drug reactions in medication treatment of type 2 diabetes
Eghbali‐Zarch, M., Tavakkoli-Moghaddam, R. R., Esfahanian, F., Amir Azaron, B., & Sepehri, M. M. (2019) · Proceedings of the Institution of Mechanical Engineers Part H Journal of Engineering in Medicine · doi:10.1177/0954411919853394 · Cited by 19
Markov Decision Process Modeling for Multi-stage Optimization of Intervention and Treatment Strategies in Breast Cancer
Imani, F., Qiu, Z., & Yang, H. Y. (2020) · Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) · doi:10.1109/embc44109.2020.9175905 · Cited by 19
Modeling hospital discharge policies for patients with pneumonia-related sepsis
Kreke, J. E., Bailey, M. D., Schaefer, A. J., Angus, D. C., & Roberts, M. S. (2008) · IIE Transactions · doi:10.1080/07408170802165872 · Cited by 19
Self-citation
A defender-attacker model and algorithm for maximizing weighted expected hitting time with application to conservation planning
Sefair, J. A., Smith, J. C., Acevedo, M. A., & Fletcher, R. J. (2017) · IISE Transactions · doi:10.1080/24725854.2017.1360533 · Cited by 18
Constrained Markov Decision Process Modeling for Optimal Sensing of Cardiac Events in Mobile Health
Yao, B., Chen, Y., & Yang, H. Y. (2021) · IEEE Transactions on Automation Science and Engineering · doi:10.1109/tase.2021.3052483 · Cited by 17
A Modeling Framework for Control of Preventive Services
Örmeci, E. L., Güneş, E. D., & Kunduzcu, D. (2015) · Manufacturing & Service Operations Management · doi:10.1287/msom.2015.0556 · Cited by 17
A Markov Decision Process Model for Cervical Cancer Screening Policies in Colombia
Akhavan‐Tabatabaei, R., Sánchez, D. M., & Yeung, T. G. (2016) · Medical Decision Making · doi:10.1177/0272989x16670622 · Cited by 16
Red Blood Cell Transfusion Safety: Probabilistic Risk Assessment and Cost/ Benefits of Risk Reduction Strategies
Slonim, A. D., Bish, E. K., & Xie, R. S. (2011) · Annals of Operations Research · doi:10.1007/s10479-011-0925-0 · Cited by 15
A finite-horizon Markov decision process model for cancer chemotherapy treatment planning: an application to sequential treatment decision making in clinical trials
Bazrafshan, N., & lotfi, M. M. (2020) · Annals of Operations Research · doi:10.1007/s10479-020-03706-5 · Cited by 15
A decision-making framework for adaptive pain management
Lin, C.-F., LeBoulluec, A. K., Zeng, L., Chen, V. C. P., & Gatchel, R. J. (2013) · Health Care Management Science · doi:10.1007/s10729-013-9252-0 · Cited by 15
Reinforcement learning approach to individualization of chronic pharmacotherapy
Gaweda, A. E., Muezzinoglu, M. K., Aronoff, G. R., Jacobs, A. A., Żurada, J. M., & Brier, M. E. (2006) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2005.1556455 · Cited by 15
Optimally solving Markov decision processes with total expected discounted reward function: Linear programming revisited
Alagöz, O., Ayvaci, M., & Linderoth, J. (2015) · Computers & Industrial Engineering · doi:10.1016/j.cie.2015.05.031 · Cited by 15
Cost effectiveness of different screening strategies for gestational diabetes mellitus screening: study protocol of a randomized community non-inferiority trial
Tehrani, F. R., Behboudi‐Gandevani, S., Abedini, M., Soleymani-Dodaran, M., Khalili, D., Farzadfar, F., Hosseinpanah, F., Hadaegh, F., Valizadeh, M., Torkestani, F., et al. (2019) · Diabetology & Metabolic Syndrome · doi:10.1186/s13098-019-0493-z · Cited by 13
Data-driven dynamic treatment planning for chronic diseases
Naumzik, C., Feuerriegel, S., & Nielsen, A. M. (2022) · European Journal of Operational Research · doi:10.1016/j.ejor.2022.06.016 · Cited by 12
A modeling framework for replacing medical therapies
Shechter, S. M., Bailey, M. D., & Schaefer, A. J. (2008) · IIE Transactions · doi:10.1080/07408170802165898 · Cited by 12
Self-citation
A Markov decision process approach to optimizing cancer therapy using multiple modalities
Maass, K. L., & Kim, M. (2019) · Mathematical Medicine and Biology A Journal of the IMA · doi:10.1093/imammb/dqz004 · Cited by 12
Markov Reward Models and Markov Decision Processes in Discrete and Continuous Time: Performance Evaluation and Optimization
Gouberman, A., & Siegle, M. (2014) · Lecture notes in computer science · doi:10.1007/978-3-662-45489-3_6 · Cited by 12
An Intelligent System for Diagnosing Thyroid Disease in Pregnant Ladies through Artificial Neural NetworkPreprint
Bhatt, V., & Pal, V. K. (2019) · SSRN Electronic Journal · doi:10.2139/ssrn.3382654 · Cited by 12
Irreversible treatment decisions under consideration of the research and development pipeline for new therapies
Shechter, S. M., Alagöz, O., & Roberts, M. S. (2010) · IIE Transactions · doi:10.1080/07408170903468589 · Cited by 10
Self-citation
Case studies of clinical decision-making through prescriptive models based on machine learning
Hoyos, W., Aguilar, J., Raciny-Aleman, M., & Toro, M. (2023) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2023.107829 · Cited by 10
Sensitivity Analysis of POMDP Value Functions
Ross, S., Izadi, M., Mercer, M., & Buckeridge, D. L. (2009) · International Conference on Machine Learning and Applications · doi:10.1109/icmla.2009.36 · Cited by 9
Kidney-related operations research: A review
Fathi, M., & Khakifirooz, M. (2019) · IISE Transactions on Healthcare Systems Engineering · doi:10.1080/24725579.2019.1640318 · Cited by 9
A Stochastic Game Analysis of Incentives and Behavioral Barriers in Chronic Disease Management
Hui, Z., Wernz, C., & Hughes, D. R. (2018) · Service Science · doi:10.1287/serv.2018.0211 · Cited by 9
A Multi-Fidelity Rollout Algorithm for Dynamic Resource Allocation in Population Disease Management
Ho, T.-Y., Liu, S., & Zabinsky, Z. B. (2018) · Health Care Management Science · doi:10.1007/s10729-018-9454-6 · Cited by 9
Minimax Regret Optimisation for Robust Planning in Uncertain Markov Decision Processes
Rigter, M. P., Lacerda, B., & Hawes, N. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i13.17417 · Cited by 3
A Markov decision process model to guide treatment of abdominal aortic aneurysms
Mattila, R., Siika, A., Roy, J., & Wahlberg, B. (2016) · The proceedings of the IEEE Conference on Control Applications/The proceedings of the ... IEEE Conference on Control Applications · doi:10.1109/cca.2016.7587869 · Cited by 9
Modeling disease management decisions for patients with pneumonia-related sepsis
Kreke, J. E. (2007) · D-Scholarship@Pitt (University of Pittsburgh) · Cited by 9
Interpretable Machine Learning for Resource Allocation with Application to Ventilator TriagePreprint
Grand-Clément, J., Goh, Y. H., Chan, C. W., Goyal, V., & Chuang, E. J. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.10994 · Cited by 9
Markov models for clinical decision‐making in radiation oncology: A systematic review
McCullum, L. B., Karagoz, A., Dede, C., Garcia, R. V., Nosrat, F., Hemmati, M., Hosseinian, S., Schaefer, A. J., Fuller, C. D., & MD Anderson Head and Neck Cancer Symptom Working Group (2024) · Journal of Medical Imaging and Radiation Oncology · doi:10.1111/1754-9485.13656 · Cited by 8
Self-citation
Optimizing Cancer Screening Using Partially Observable Markov Decision Processes
Alagöz, O. (2011) · doi:10.1287/educ.1110.0087 · Cited by 8
Stochastic Model Checking. Rigorous Dependability Analysis Using Model Checking Techniques for Stochastic Systems
Remke, A., & Stoelinga, M. I. A. (2014) · Lecture notes in computer science · doi:10.1007/978-3-662-45489-3 · Cited by 8
Bayesian learning of dose–response parameters from a cohort under response-guided dosing
Kotas, J., & Ghate, A. (2015) · European Journal of Operational Research · doi:10.1016/j.ejor.2017.07.034 · Cited by 8
Population-level intervention and information collection in dynamic healthcare policy
Cipriano, L. E., & Weber, T. A. (2017) · Health Care Management Science · doi:10.1007/s10729-017-9415-5 · Cited by 8
Optimal action sequence generation for assistive agents in fixed horizon tasks
Baraka, K., Melo, F. S., Couto, M., & Veloso, M. (2020) · Autonomous Agents and Multi-Agent Systems · doi:10.1007/s10458-020-09458-7 · Cited by 8
WHEN TO INITIATE, WHEN TO SWITCH, AND HOW TO SEQUENCE HIV THERAPIES: A MARKOV DECISION PROCESS APPROACH
Shechter, S. M. (2006) · D-Scholarship@Pitt (University of Pittsburgh) · Cited by 8
Self-citation
Generating Reward Functions Using IRL Towards Individualized Cancer Screening
Petousis, P., Han, S., Hsu, W. W. Y., & Bui, A. A. T. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-12738-1_16 · Cited by 7
Information Relaxation Bounds for Partially Observed Markov Decision Processes
Haugh, M. B., & Lacedelli, O. R. (2019) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2019.2942528 · Cited by 7
Response-guided dosing for rheumatoid arthritis
Kotas, J., & Ghate, A. (2015) · IIE Transactions on Healthcare Systems Engineering · doi:10.1080/19488300.2015.1126873 · Cited by 7
A MDP solution for Traditional Chinese medicine treatment planning
Feng, Q., Zhou, X., Huang, H., Yu, J. Q., Zhang, Y., Tong, X., Zhang, R., Wang, Y., & Liu, B. (2010) · International Conference on Biomedical Engineering and Informatics (BMEI) · doi:10.1109/bmei.2010.5639423 · Cited by 7
Health System Innovation: Analytics in Action
Copenhaver, M. S., Hu, M., Levi, R., Safavi, K. C., & Langle, A. C. Z. (2019) · doi:10.1287/educ.2019.0202 · Cited by 6
Approximate dynamic programming for anemia management.
Muezzinoglu, M. K. (2006) · doi:10.18297/etd/1024 · Cited by 6
Personalized medication recommendations for Parkinson’s disease patients using gated recurrent units and SHAP interpretability
Riasi, A., Delrobaei, M., & Salari, M. (2025) · Scientific Reports · doi:10.1038/s41598-025-04217-8 · Cited by 5
Optimal discharge of patients from intensive care via a data-driven policy learning framework
Lejarza, F., Calvert, J. S., Attwood, M. M., Evans, D. A., & Mao, Q. (2023) · Operations Research for Health Care · doi:10.1016/j.orhc.2023.100400 · Cited by 5
Cost-effectiveness analysis of surgical proximal femur fracture prevention in elderly: a Markov cohort simulation model
Alnemer, M. S., Kotliar, K. E., Neuhaus, V., Pape, H., & Ciritsis, B. D. (2023) · Cost Effectiveness and Resource Allocation · doi:10.1186/s12962-023-00482-4 · Cited by 5
Multimodality treatment planning using the Markov decision process: a comprehensive study of applications and challenges
Singh, P., Singh, S., Mishra, A., & Mishra, S. K. (2024) · Research on Biomedical Engineering · doi:10.1007/s42600-024-00349-4 · Cited by 5
Partially Observable Markov Decision Processes for Prostate Cancer Screening
Jing-yu, Z. (2011) · NCSU Libraries Repository (North Carolina State University Libraries) · Cited by 5
Using a game-theoretic approach to design optimal health insurance for chronic disease
Ho, T.-Y., Fishman, P. A., & Zabinsky, Z. B. (2019) · IISE Transactions on Healthcare Systems Engineering · doi:10.1080/24725579.2019.1567626 · Cited by 5
Mining clinical pathways for daily insulin therapy of diabetic children
Deja, R., Froelich, W., & Deja, G. (2021) · International Journal of Applied Mathematics and Computer Science · doi:10.34768/amcs-2021-0008 · Cited by 5
Extubation Decisions with Predictive Information for Mechanically Ventilated Patients in the ICU
Cheng, G., Xie, J., Zheng, Z., Luo, H., & Ooi, O. C. (2019) · Management Science · doi:10.1287/mnsc.2021.01427 · Cited by 4
Healthcare Optimization : Current Status and Vitalization Suggestions
Kang, S.-H., Kim, B., Jun, C., Choi, B. K., & Lee, S. (2013) · Journal of Korean Institute of Industrial Engineers · doi:10.7232/jkiie.2013.39.4.313 · Cited by 4
Robust topological policy iteration for infinite horizon bounded Markov Decision Processes
Reis, W. A. S., de Barros, L. N., & Delgado, K. V. (2018) · International Journal of Approximate Reasoning · doi:10.1016/j.ijar.2018.12.004 · Cited by 4
Heuristic algorithm for nested Markov decision process: Solution quality and computational complexity
Fianu, S., & Davis, L. (2023) · Computers & Operations Research · doi:10.1016/j.cor.2023.106297 · Cited by 4
Quantifying the benefits of increasing decision-making frequency for health applications with regular decision epochs
Zhang, S., Suen, S., Sundaram, V., & Gong, C. L. (2024) · IISE Transactions · doi:10.1080/24725854.2024.2321492 · Cited by 4
Optimal Decision Making for Breast Cancer Treatment in the Presence of Cancer Regression and Type II Error in Mammography Results
Vargas, S. A., Zhang, S., & Akhavan‐Tabatabaei, R. (2015) · Springer proceedings in mathematics & statistics · doi:10.1007/978-3-319-12583-1_13 · Cited by 4
Personalized Medicine in Chronic Disease Management.
Schell, G. J. (2015) · Deep Blue (University of Michigan) · Cited by 4
Markov Decision Processes
Scheller-Wolf, A. A. (2018) · Wiley eBooks · doi:10.1002/9781119300977.ch14 · Cited by 4
Approximate Dynamic Programming for Commodity and Energy Merchant Operations
Nadarajah, S. (2014) · Figshare · doi:10.1184/r1/6714824 · Cited by 4

Authors: Schaefer, A. J., Bailey, M. D., Shechter, S. M., & Roberts, M. S.
Affiliations: University of Pittsburgh (US)
ORCID: Schaefer, Andrew J.
Bailey, Matthew D.
Shechter, Steven M.
Roberts, Mark S.
Venue: International Series in Operations Research & Management Science
Publisher: Springer US
Published: 2005
Pages: 593-612
ISSN: 0884-8289 (ISSN-L)
DOI: 10.1007/1-4020-8066-2_23
OpenAlex: W58990062
Cited by: 155
References: 53
Citations by year: 2026: 3 · 2025: 7 · 2024: 11 · 2023: 13 · 2022: 6 · 2021: 13 · 2020: 8 · 2019: 17 · 2018: 15 · 2017: 9 · 2016: 5 · 2015: 12 · 2014: 3 · 2013: 6 · 2012: 5
FWCI: 8.16
Type: book chapter
Language: en
Open access: closed
Licence: www.springernature.com/gp/researchers/text-and-data-mining (tdm)
www.springernature.com/gp/researchers/text-and-data-mining (vor)
Locations: International series in management science/operations research/International series in operations research & management science (publishedVersion)
Funders: National Science Foundation
Topics: Social Sciences › Economics, Econometrics and Finance › Economics and Econometrics › Health Systems, Economic Evaluations, Quality of Life
Social Sciences › Decision Sciences › Management Science and Operations Research › Complex Systems and Decision Making
Physical Sciences › Engineering › Control and Systems Engineering › Adaptive Dynamic Programming Control
Keywords: Markov decision process, treatment decision-making, sequential decision making, stochastic dynamic programming, treatment planning, clinical decision support

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Modeling Medical Treatment Using Markov Decision Processes

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Citations

Cites

Cited by

Related

Metadata

Record

About these checks