Evaluating Systems Pharmacology Models Is Different From Evaluating Standard Pharmacokinetic–Pharmacodynamic Models
Open accessAbstract
Based on the author's recent experience, there appears to be some confusion regarding the steps required to qualify a systems pharmacology model as adequate for the intended purpose. This manuscript outlines the model evaluation approach used in the author's recent publication1 on the systems pharmacology of a 5‐lipoxygenase inhibitor and is an attempt to generate discussion on this topic within the pharmacometrics and systems pharmacology community. CPT Pharmacometrics Syst. Pharmacol. (2014) 3, e101; doi:10.1038/psp.2013.77; published online 19 February 2014
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Model quality tests such as Akaike's information criterion, Bayesian information criterion, R2, and Q2 may be inadequate for QSP because the goal of QSP models is not to obtain a highly precise estimate of system readouts but to obtain a ballpark estimate with limited knowledge of the system components.
“Model quality tests usually run in the context of other in silico simulations, such as Akaike's information criterion, Bayesian information criterion, R 2, Q 2, and so on may be inadequate here because the goal of QSP models is not to obtain a highly precise estimate of the system readouts (e.g., biomarker levels changes), but to obtain a ballpark estimate with the limited knowledge that there is”
From Best Practices to Maximize the Use and Reuse of Quantitative and Systems Pharmacology Models: Recommendations From the United Kingdom Quantitative and Systems Pharmacology Network (2019) · cites this paper as [78]
Split into 2 atoms — 0 confirmed, 2 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.
Experimental — accuracy not yet measuredEntelos QSP simulations for a first-in-class therapy against type 2 diabetes mellitus led to a reduction from a 6-arms-for-14-weeks design to a 2-arms-for-8-weeks phase I clinical trial design.
“A good example of the latter was reported by Entelos80 in their publication78 about the optimization of a phase I clinical trial protocol for a first‐in‐class therapy against type 2 diabetes mellitus (T2DM): their QSP simulations led to the reduction from a “6 arms for 14 weeks” to a “2 arms for 8 weeks” design.”
From Best Practices to Maximize the Use and Reuse of Quantitative and Systems Pharmacology Models: Recommendations From the United Kingdom Quantitative and Systems Pharmacology Network (2019) · cites this paper as [78]
The ‘Clear Statement of the Objective of the Modeling Exercise’ section mentions study design only as a possible quantitative objective; the ‘Discussion and Conclusion’ section discusses evaluation of a 5-lipoxygenase model. Neither passage reports an Entelos diabetes trial or the claimed change in arms and duration. Searches for Entelos, the phase I trial, and the design figures found no supporting passage in our copy (pages not specified). (LLM, conf=0.76)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Nyman, E., Brännmark, C., Palmér, R., Brugård, J., Nyström, F. H., Strålfors, P., & Cedersund, G. (2011) · Journal of Biological Chemistry · doi:10.1074/jbc.m110.188987 · Cited by 77 · see citing works
Negative Modeling Results: A Dime a Dozen or a Stepping Stone to Scientific Discovery?
Hendriks, B. (2013) · CPT: Pharmacometrics & Systems Pharmacology · doi:10.1038/psp.2013.28 · Cited by 6 · see citing works
Flagged first, then by citations
Research works printed but not linked: 6
- Demin O. Systems pharmacology models can be used to understand complex pharmacokinetic-pharmacodynamic behavior: an example using 5-lipoxygenase inhibitors. CPT Pharmacometrics Syst. Pharmacol 2 e74 2013 24026253 PMC4026633
- Brännmark C. Insulin signaling in type 2 diabetes: experimental and modeling analyses reveal mechanisms of insulin resistance in human adipocytes. J. Biol. Chem 288 9867 9880 2013 23400783 PMC3617287
- Landersdorfer C.B. He Y.L. Jusko W.J. Mechanism-based population modelling of the effects of vildagliptin on GLP-1, glucose and insulin in patients with type 2 diabetes. Br. J. Clin. Pharmacol 73 373 390 2012 22442825 PMC3370342
- Peterson M.C. Riggs M.M. Predicting nonlinear changes in bone mineral density over time using a multiscale systems pharmacology model. CPT Pharmacometrics Syst. Pharmacol 1 e14 2012 23835796 PMC3600731
- Toghaw P. Matone A. Lenbury Y. De Gaetano A. Bariatric surgery and T2DM improvement mechanisms: a mathematical model. Theor. Biol. Med. Model 9 16 2012 22587410 PMC3586953
- Cedersund G. Strålfors P. Putting the pieces together in diabetes research: towards a hierarchical model of whole-body glucose homeostasis. Eur. J. Pharm. Sci 36 91 104 2009 19056492
Cited by
Sorted by citations
Jones, H., Chen, Y., Gibson, C., Heimbach, T., Parrott, N., Peters, S., Snoeys, J., Upreti, V., Zheng, M., & Hall, S. (2015) · Clinical Pharmacology & Therapeutics · doi:10.1002/cpt.37 · Cited by 488 · see citing works
Zhang, X., Trame, M., Lesko, L., & Schmidt, S. (2015) · CPT: Pharmacometrics & Systems Pharmacology · doi:10.1002/psp4.6 · Cited by 467 · see citing works
Shebley, M., Sandhu, P., Emami Riedmaier, A., Jamei, M., Narayanan, R., Patel, A., Peters, S. A., Reddy, V. P., Zheng, M., de Zwart, L., et al. (2018) · Clinical Pharmacology & Therapeutics · doi:10.1002/cpt.1013 · Cited by 395 · see citing works
Jamei, M. (2016) · Current Pharmacology Reports · doi:10.1007/s40495-016-0059-9 · Cited by 244 · see citing works
A Six‐Stage Workflow for Robust Application of Systems Pharmacology
Gadkar, K., Kirouac, D., Mager, D., van der Graaf, P., & Ramanujan, S. (2016) · CPT: Pharmacometrics & Systems Pharmacology · doi:10.1002/psp4.12071 · Cited by 93 · see citing works
Reverse Translation in PBPK and QSP: Going Backwards in Order to Go Forward With Confidence
Rostami‐Hodjegan, A. (2018) · Clinical Pharmacology & Therapeutics · doi:10.1002/cpt.904 · Cited by 90 · see citing works
Friedrich, C. (2016) · CPT: Pharmacometrics & Systems Pharmacology · doi:10.1002/psp4.12056 · Cited by 87 · see citing works
Cucurull‐Sanchez, L., Chappell, M. J., Chelliah, V., Amy Cheung, S. Y., Derks, G., Penney, M., Phipps, A., Malik‐Sheriff, R. S., Timmis, J., Tindall, M. J., et al. (2019) · CPT: Pharmacometrics & Systems Pharmacology · doi:10.1002/psp4.12381 · Cited by 61 · see citing works
“, biomarker levels changes), but to obtain a ballpark estimate with the limited knowledge that there is of the system components, of what can be expected when all of those components are put together.(78)”
BackgroundCheng, Y., Thalhauser, C. J., Smithline, S., Pagidala, J., Miladinov, M., Vezina, H. E., Gupta, M., Leil, T. A., & Schmidt, B. J. (2017) · The AAPS Journal · doi:10.1208/s12248-017-0100-x · Cited by 54 · see citing works
- Authors
- Agoram, B.
- Affiliations
- MedImmune LLC, Clinical Pharmacology, Drug Metabolism, and Pharmacokinetics Cambridge UK
- Corresponding author
- Agoram, B
- Venue
- CPT: Pharmacometrics & Systems Pharmacology
- Publisher
- Wiley
- Published
- 2014-02
- Volume
- 3
- Issue
- 2
- Pages
- 1-3
- ISSN
- 2163-8306 (ISSN-L)
- PMC
- PMC3944115
- OpenAlex
- W2077685839
- Cited by
- 30
- References
- 2
- Citations by year
- 2024: 1 · 2022: 2 · 2021: 3 · 2020: 2 · 2019: 2 · 2018: 2 · 2017: 2 · 2016: 9 · 2015: 5 · 2014: 2
- FWCI
- 6.33
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- CPT Pharmacometrics & Systems Pharmacology (publishedVersion, cc-by-nc-nd)
- Europe PMC (PubMed Central) (publishedVersion, cc-by-nc-nd)
- PubMed Central (publishedVersion, cc-by-nc-nd)
- PubMed (publishedVersion)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Health Sciences › Medicine › Pediatrics, Perinatology and Child Health › Pharmaceutical studies and practices
- Physical Sciences › Computer Science › Computational Theory and Mathematics › Computational Drug Discovery Methods
- Keywords
- model evaluation, model qualification, pharmacokinetic-pharmacodynamic modeling, 5-lipoxygenase inhibitors, model adequacy
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
