Reinforcement learning as an innovative model-based approach: Examples from precision dosing, digital health and computational psychiatry
Open accessAbstract
Model-based approaches are instrumental for successful drug development and use. Anchored within pharmacological principles, through mathematical modeling they contribute to the quantification of drug response variability and enables precision dosing. Reinforcement learning (RL)—a set of computational methods addressing optimization problems as a continuous learning process—shows relevance for precision dosing with high flexibility for dosing rule adaptation and for coping with high dimensional efficacy and/or safety markers, constituting a relevant approach to take advantage of data from digital health technologies. RL can also support contributions to the successful development of digital health applications, recognized as key players of the future healthcare systems, in particular for reducing the burden of non-communicable diseases to society. RL is also pivotal in computational psychiatry—a way to characterize mental dysfunctions in terms of aberrant brain computations—and represents an innovative modeling approach forpsychiatric indications such as depression or substance abuse disorders for which digital therapeutics are foreseen as promising modalities.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The indicated BBB penetration values are inconsistent with the physicochemical characteristics of monoclonal antibodies (solanezumab, aducanumab, bapineuzumab) or gamma-secretase inhibitors (semagacestat, tarenflurbil).
“The indicated BBB penetration values are inconsistent with the physicochemical characteristics of monoclonal antibodies (solanezumab, aducanumab, bapineuzumab) or gamma-secretase inhibitors (semagacestat, tarenflurbil) [193,194,195].”
From Recent Advances in Therapeutics for the Treatment of Alzheimer’s Disease (2024) · cites this paper as [195]
The BBB is impermeable to 98% of small molecules and around 100% of large molecules in relevant quantities.
“The BBB is impermeable to 98% of small molecules and around 100% of large molecules in relevant quantities.”
From Recent Advances in Therapeutics for the Treatment of Alzheimer’s Disease (2024) · cites this paper as [195]
For a medicine to penetrate the BBB, its physicochemical properties-polar surface area, molecular weight, and lipophilicity as assessed by hydrogen bonding ability-are essential.
“For a medicine to penetrate the blood–brain barrier, its physicochemical properties—such as its polar surface area, molecular weight, and lipophilicity as assessed by hydrogen bonding ability—are essential.”
From Recent Advances in Therapeutics for the Treatment of Alzheimer’s Disease (2024) · cites this paper as [195]
Candidates for BBB penetration are characterized by molecular weight less than 450 Daltons, polar surface area less than 90 Å2, and hydrogen bond donor number less than 3.
“A molecular weight less than 450 Daltons, a polar surface area less than 90 Å2, and a hydrogen bond donor number less than 3 are characteristics of candidates for BBB penetration.”
From Recent Advances in Therapeutics for the Treatment of Alzheimer’s Disease (2024) · cites this paper as [195]
The blood-brain barrier (BBB) is an important impediment to the development of AD medications because it prevents potential drug candidates from entering the brain parenchyma.
“An important impediment to the development of AD medications is the blood–brain barrier (BBB), which prevents potential drug candidates from entering the brain parenchyma.”
From Recent Advances in Therapeutics for the Treatment of Alzheimer’s Disease (2024) · cites this paper as [195]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Reinforcement Learning: An Introduction
Johnson, J. D., Li, J., & Chen, Z. (2000) · Neurocomputing · doi:10.1016/s0925-2312(00)00324-6 · Cited by 8,821 · see citing works
Lee, I.-M., Shiroma, E. J., Lobelo, F., Puska, P., Blair, S. N., & Katzmarzyk, P. T. (2012) · The Lancet · doi:10.1016/s0140-6736(12)61031-9 · Cited by 8,215 · see citing works
Insel, T., Cuthbert, B., Garvey, M., Heinssen, R., Pine, D. S., Quinn, K., Sanislow, C., & Wang, P. (2010) · American Journal of Psychiatry · doi:10.1176/appi.ajp.2010.09091379 · Cited by 7,067 · see citing works
Fit Yourself and Take Your Lungs to Heart
Palange, P., & Bonini, M. (2013) · Respiration · doi:10.1159/000350831 · Cited by 2,780 · see citing works
Nahum-Shani, I., Smith, S. N., Spring, B. J., Collins, L. M., Witkiewitz, K., Tewari, A., & Murphy, S. A. (2018) · Annals of Behavioral Medicine · doi:10.1007/s12160-016-9830-8 · Cited by 2,171 · see citing works
Insel, T. R., & Cuthbert, B. N. (2015) · Science · doi:10.1126/science.aab2358 · Cited by 840 · see citing works
Montague, P. R., Dolan, R. J., Friston, K. J., & Dayan, P. (2012) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2011.11.018 · Cited by 809 · see citing works
Microrandomized trials: An experimental design for developing just-in-time adaptive interventions.
Klasnja, P., Hekler, E. B., Shiffman, S., Boruvka, A., Almirall, D., Tewari, A., & Murphy, S. A. (2015) · Health Psychology · doi:10.1037/hea0000305 · Cited by 671 · see citing works
A Tutorial on Thompson Sampling
Russo, D., Roy, B. V., Kazerouni, A., Osband, I., & Wen, Z. (2018) · Foundations and Trends® in Machine Learning · doi:10.1561/2200000070 · Cited by 543 · see citing works
Model of Chemotherapy-Induced Myelosuppression With Parameter Consistency Across Drugs
Friberg, L. E., Henningsson, A., Maas, H. J., et al. (2002) · Journal of Clinical Oncology · doi:10.1200/jco.2002.02.140 · Cited by 537 · see citing works
Addiction as a Computational Process Gone Awry
Redish, A. D. (2004) · Science · doi:10.1126/science.1102384 · Cited by 534 · see citing works
Learning to Optimize via Posterior Sampling
Russo, D. J., & Roy, B. V. (2014) · Mathematics of Operations Research · doi:10.1287/moor.2014.0650 · Cited by 526 · see citing works
Mapping anhedonia onto reinforcement learning: a behavioural meta-analysis
Huys, Q. J., Pizzagalli, D. A., Bogdan, R., & Dayan, P. (2013) · Biology of Mood & Anxiety Disorders · doi:10.1186/2045-5380-3-12 · Cited by 480 · see citing works
Crocker, L. D., Heller, W., Warren, S. L., O'Hare, A. J., Infantolino, Z. P., & Miller, G. A. (2013) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2013.00261 · Cited by 193 · see citing works
Mixed Effects Models for the Population Approach
Lavielle, M. (2014) · Chapman & Hall/CRC eBooks · doi:10.1201/b17203 · Cited by 183 · see citing works
Liao, P., Greenewald, K. H., Klasnja, P., & Murphy, S. A. (2020) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3381007 · Cited by 152 · see citing works
Digital Therapeutics: An Integral Component of Digital Innovation in Drug Development
Sverdlov, O., van Dam, J., Hannesdottir, K., & Thornton‐Wells, T. (2018) · Clinical Pharmacology & Therapeutics · doi:10.1002/cpt.1036 · Cited by 147 · see citing works
A Tumor Growth Inhibition Model for Low-Grade Glioma Treated with Chemotherapy or Radiotherapy
Ribba, B., Kaloshi, G., Peyre, M., et al. (2012) · Clinical Cancer Research · doi:10.1158/1078-0432.ccr-12-0084 · Cited by 141 · see citing works
Self-citationLu, J. J.-Q., Deng, K., Zhang, X., Liu, G., & Guan, Y. (2021) · iScience · doi:10.1016/j.isci.2021.102804 · Cited by 127 · see citing works
Qian, T., Walton, A. E., Collins, L. M., Klasnja, P., Lanza, S. T., Nahum-Shani, I. B., Rabbi, M., Russell, M. A., Walton, M. A., Yoo, H., et al. (2022) · Psychological Methods · doi:10.1037/met0000283 · Cited by 108 · see citing works
(2016) · The MIT Press eBooks · doi:10.7551/mitpress/9780262035422.001.0001 · Cited by 72 · see citing works
Yauney, G., & Shah, P. (2018) · Machine Learning for Healthcare Conference · Cited by 52 · see citing works
Geerts, H., Wikswo, J. P., van der Graaf, P. H., Bai, J. P. F., Gaiteri, C., Bennett, D. A., Swalley, S. E., Schuck, E. L., Kaddurah‐Daouk, R., Tsaioun, K., et al. (2019) · CPT Pharmacometrics & Systems Pharmacology · doi:10.1002/psp4.12478 · Cited by 49 · see citing works
Maier, C., Hartung, N., Kloft, C., Huisinga, W., & de Wiljes, J. (2021) · CPT Pharmacometrics & Systems Pharmacology · doi:10.1002/psp4.12588 · Cited by 48 · see citing works
Precision Dosing: An Industry Perspective
Peck, R. (2020) · Clinical Pharmacology & Therapeutics · doi:10.1002/cpt.2064 · Cited by 36 · see citing works
Introduction of an artificial neural network–based method for concentration‐time predictions
Bräm, D. S., Parrott, N. J., Hutchinson, L. G., & Steiert, B. (2022) · CPT Pharmacometrics & Systems Pharmacology · doi:10.1002/psp4.12786 · Cited by 36 · see citing works
Qian, T., Klasnja, P., & Murphy, S. A. (2020) · Statistical Science · doi:10.1214/19-sts720 · Cited by 28 · see citing works
Hénin, É., Meille, C., Barbolosi, D., et al. (2016) · Breast Cancer Research and Treatment · doi:10.1007/s10549-016-3760-9 · Cited by 25 · see citing works
Ribba, B., Bräm, D. S., Baverel, P., et al. (2022) · CPT Pharmacometrics & Systems Pharmacology · doi:10.1002/psp4.12858 · Cited by 23 · see citing works
Self-citationQuantitative systems pharmacology in neuroscience: Novel methodologies and technologies
Bloomingdale, P., Karelina, T., Cirit, M., Muldoon, S. F., Baker, J. T., McCarty, W. J., Geerts, H., & Macha, S. (2021) · CPT Pharmacometrics & Systems Pharmacology · doi:10.1002/psp4.12607 · Cited by 22 · see citing works
Conrado, D. J., Duvvuri, S., Geerts, H., Burton, J., Biesdorf, C., Ahamadi, M., Macha, S., Hather, G., Francisco Morales, J., Podichetty, J., et al. (2020) · Clinical Pharmacology & Therapeutics · doi:10.1002/cpt.1782 · Cited by 21 · see citing works
Wang, D., Song, Z., Zhang, C., et al. (2021) · Medicine · doi:10.1097/md.0000000000023930 · Cited by 20 · see citing works
Maxfield, K., & Zineh, I. (2021) · JAMA · doi:10.1001/jama.2021.1004 · Cited by 20 · see citing works
Paradigms for the development of transformative medicines—lessons from the EGFR story
Pao, W. J., & Nagel, Y. A. (2022) · Annals of Oncology · doi:10.1016/j.annonc.2022.02.005 · Cited by 4 · see citing works
Flagged first, then by citations
Research works printed but not linked: 1
- Jacobson N. C. Kowatsch T. Marsch L. A. (2022). Digital therapeutics for mental health and addiction: The state of the science and vision for the future. San Diego, CA: Academic Press, 270.
Cited by
Sorted by citations
Recent Advances in Therapeutics for the Treatment of Alzheimer’s Disease
Thawabteh, A. M., Ghanem, A. W., AbuMadi, S., Thaher, D., Jaghama, W., Karaman, D., & Karaman, R. (2024) · Molecules · doi:10.3390/molecules29215131 · Cited by 56 · see citing works
“The indicated BBB penetration values are inconsistent with the physicochemical characteristics of monoclonal antibodies (solanezumab, aducanumab, bapineuzumab) or gamma-secretase inhibitors (semagacestat, tarenflurbil) [193–195].”
- Authors
- Ribba, B.
- Affiliations
- F. Hoffmann-La Roche AG (CH)
- Corresponding author
- Ribba, Benjamin
- Venue
- Frontiers in Pharmacology
- Publisher
- Frontiers Media SA
- Published
- 2023-02-17
- Volume
- 13
- Pages
- 1094281-1094281
- ISSN
- 1663-9812 (ISSN-L)
- PMC
- PMC9981647
- OpenAlex
- W4321238202
- Cited by
- 24
- References
- 34
- Citations by year
- 2026: 6 · 2025: 7 · 2024: 9 · 2023: 2
- FWCI
- 4.87
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Frontiers in Pharmacology (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Keywords
- reinforcement learning, model-based drug development, precision dosing, digital health, computational psychiatry, digital therapeutics, drug response variability, non-communicable diseases, depression, substance use disorders
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
