Inverse reinforcement learning for intelligent mechanical ventilation and sedative dosing in intensive care units
Open accessThe DOI matches its Crossref record; 2 of the 2 references it cites were found in a registry or an index.
Abstract
BACKGROUND: Reinforcement learning (RL) provides a promising technique to solve complex sequential decision making problems in health care domains. To ensure such applications, an explicit reward function encoding domain knowledge should be specified beforehand to indicate the goal of tasks. However, there is usually no explicit information regarding the reward function in medical records. It is then necessary to consider an approach whereby the reward function can be learned from a set of presumably optimal treatment trajectories using retrospective real medical data. This paper applies inverse RL in inferring the reward functions that clinicians have in mind during their decisions on weaning of mechanical ventilation and sedative dosing in Intensive Care Units (ICUs). METHODS: We model the decision making problem as a Markov Decision Process, and use a batch RL method, Fitted Q Iterations with Gradient Boosting Decision Tree, to learn a suitable ventilator weaning policy from real trajectories in retrospective ICU data. A Bayesian inverse RL method is then applied to infer the latent reward functions in terms of weights in trading off various aspects of evaluation criterion. We then evaluate how the policy learned using the Bayesian inverse RL method matches the policy given by clinicians, as compared to other policies learned with fixed reward functions. RESULTS: ) which is supported by previous RL methods. Moreover, by discovering the optimal weights, new effective treatment protocols can be suggested. CONCLUSIONS: Inverse RL is an effective approach to discovering clinicians' underlying reward functions for designing better treatment protocols in the ventilation weaning and sedative dosing in future ICUs.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
(2000) · doi:10.2460/ajvr.67.2.323 · Cited by 1,621 · cited 2×
“Ng and Russell first introduced IRL to describe the problem of recovering a reward function of an MDP from demonstrations [10].”
BackgroundOptimal medication dosing from suboptimal clinical examples: a deep reinforcement learning approach
Nemati, S., Ghassemi, M. M., & Clifford, G. D. (2016) · 2016 38th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) · doi:10.1109/embc.2016.7591355 · Cited by 195
Cited by
Reinforcement Learning in Healthcare: A Survey
Yu, C., Liu, J., Nemati, S., & Yin, G. (2023) · ACM Computing Surveys · doi:10.1145/3477600 · Cited by 628
Self-citationMachine Learning and the Future of Cardiovascular Care
Quer, G., Arnaout, R., Henne, M., & Arnaout, R. (2021) · Journal of the American College of Cardiology · doi:10.1016/j.jacc.2020.11.030 · Cited by 337
Reinforcement learning for intelligent healthcare applications: A survey
Coronato, A., Naeem, M., De Pietro, G., & Paragliola, G. (2020) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2020.101964 · Cited by 320
Application Scenarios for Artificial Intelligence in Nursing Care: Rapid Review
Seibert, K., Domhoff, D., Bruch, D., Schulte-Althoff, M., Fürstenau, D., Biessmann, F., & Wolf-Ostermann, K. (2021) · Journal of Medical Internet Research · doi:10.2196/26522 · Cited by 314
“…nurses and patient care is evident when trying to reduce false alarm rates in monitoring [71,316] or to improve mechanical ventilation and sedative dosing processes [72], there is a blurred line between AI systems to support medical diagnostics and therapy, and AI systems to support nursing care.”
Reinforcement learning for clinical decision support in critical care: comprehensive review
Liu, S., See, K. C., Ngiam, K. Y., Celi, L. A., Sun, X., & Feng, M. (2020) · Journal of Medical Internet Research · doi:10.2196/18477 · Cited by 225
“Most of the reward functions were designed a priori with guidance from clinical practice and protocols, but 2 studies [40,45] managed to directly learn the reward function from the data using inverse RL.”
MethodResultInfluential
Metadata
- Authors
- Yu, C., Liu, J., & Zhao, H.
- Venue
- BMC Medical Informatics and Decision Making
- Published
- 2019-04
- Volume
- 19
- Issue
- S2
- Pages
- 57-57
- ISSN
- 1472-6947 (ISSN-L)
- OpenAlex
- W2937671098
- Cited by
- 59
- References
- 2
- FWCI
- 2.88
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Inverse reinforcement learning for intelligent mechanical ventilation and sedative dosing in intensive care units. https://app.cochat.ai/papers/10.1186/s12911-019-0763-6
- The paper
- Yu, C., Liu, J., & Zhao, H. (2019). Inverse reinforcement learning for intelligent mechanical ventilation and sedative dosing in intensive care units. BMC Medical Informatics and Decision Making. https://doi.org/10.1186/s12911-019-0763-6
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.