The DOI matches its Crossref record; 49 of the 49 references it cites were found in a registry or an index.
In silico trials: Verification, validation and uncertainty quantification of predictive models used in the regulatory evaluation of biomedical products
Open accessAbstract
Historically, the evidences of safety and efficacy that companies provide to regulatory agencies as support to the request for marketing authorization of a new medical product have been produced experimentally, either in vitro or in vivo. More recently, regulatory agencies started receiving and accepting evidences obtained in silico, i.e. through modelling and simulation. However, before any method (experimental or computational) can be acceptable for regulatory submission, the method itself must be considered "qualified" by the regulatory agency. This involves the assessment of the overall "credibility" that such a method has in providing specific evidence for a given regulatory procedure. In this paper, we describe a methodological framework for the credibility assessment of computational models built using mechanistic knowledge of physical and chemical phenomena, in addition to available biological and physiological knowledge; these are sometimes referred to as "biophysical" models. Using guiding examples, we explore the definition of the context of use, the risk analysis for the definition of the acceptability thresholds, and the various steps of a comprehensive verification, validation and uncertainty quantification process, to conclude with considerations on the credibility of a prediction for a specific context of use. While this paper does not provide a guideline for the formal qualification process, which only the regulatory agencies can provide, we expect it to help researchers to better appreciate the extent of scrutiny required, which should be considered early on in the development/use of any (new) in silico evidence.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Of particular relevance for the field of systems biology is The European Medicines Agency’s (EMA) Guideline on the Reporting of Physiologically Based Pharmacokinetic (PBPK) Modeling and Simulation issued in 2018 [9, 65].”
From Adapting modeling and simulation credibility standards to computational systems biology (2023) · cites this paper as [65]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Verification and Validation in Computational Science and Engineering
Roache, P. J. (1998) · Cited by 2,160
Verification and Validation in Scientific Computing
Oberkampf, W. L., & Roy, C. J. (2010) · doi:10.1017/cbo9780511760396 · Cited by 1,556 · cited 3×
“The reader is referred to ASME VV-40-2018 for a complete description of all credibility factors and to ASME VV-10-2007,4 ASME VV-202009,5 Oberkampf and Roy [19], and Roache [20] for more detailed information on model verification, validation, and uncertainty quantification.”
BackgroundMethodNonlinear System Identification: NARMAX Methods in the Time, Frequency, and Spatio-Temporal Domains
Billings, S. A. (2013) · CERN Document Server (European Organization for Nuclear Research) · Cited by 655
Sager, P. T., Gintant, G. A., Turner, J. R., Pettit, S. D., & Stockbridge, N. (2013) · American Heart Journal · doi:10.1016/j.ahj.2013.11.004 · Cited by 541 · cited 1×
“CiPA has become a global effort, involving many industry and academic partners, in addition to regulators [18,3].”
BackgroundComputational models in cardiology
Niederer, S., Lumens, J., & Trayanova, N. A. (2018) · Nature Reviews Cardiology · doi:10.1038/s41569-018-0104-y · Cited by 435 · cited 1×
“Current human cardiac electrophysiology models integrate detailed information on the dynamic processes underlying cardiac electrical excitation from subcellular to whole organ levels [14].”
BackgroundExtracting clinically relevant data from finite element simulations
Viceconti, M., Olsen, S., Nolte, L.-P., & Burton, K. A. (2005) · Clinical Biomechanics · doi:10.1016/j.clinbiomech.2005.01.010 · Cited by 343 · cited 1×
“But when the predictions of a computer model are used to make clinical recommendations, there is implicit risk to the patient(s) associated with that computer model; and therefore it is necessary to demonstrate the credibility of the model is commensurate with the patient risk [48].”
MethodSelf-citationLearning under Concept Drift: an OverviewPreprint
Žliobaitė, I. (2010) · arXiv (Cornell University) · doi:10.48550/arxiv.1010.4784 · Cited by 332 · cited 1×
“Indeed, a recent FDA proposal outlining a regulatory framework for Artificial Intelligence/Machine Learning (AI/ML)-Based Software as a Medical Device (SaMD) [29] suggested that these models should be continuously tested in order to avoid issues such as context drift [30].”
BackgroundPassini, E., Britton, O. J., Lu, H. R., Rohrbacher, J., Hermans, A. N., Gallacher, D. J., Greig, R. J. H., Bueno‐Orovio, A., & Rodríguez, B. (2017) · Frontiers in Physiology · doi:10.3389/fphys.2017.00668 · Cited by 326 · cited 1×
“This class of trial exploits human-based modelling and simulation technologies for virtual testing of pharmacological therapies [7] and devices [8].”
BackgroundIn silico clinical trials: how computer simulation will transform the biomedical industry
Viceconti, M., HENNEY, A. M., & Morley‐Fletcher, E. (2016) · International Journal of Clinical Trials · doi:10.18203/2349-3259.ijct20161408 · Cited by 316
Self-citationIn silico clinical trials: concepts and early adoptions
Pappalardo, F., Russo, G. L., Musuamba, F. T., & Viceconti, M. (2018) · Briefings in Bioinformatics · doi:10.1093/bib/bby043 · Cited by 281 · cited 2×
“Thus, human in silico clinical trials are now emerging as an important paradigm in the development of medical therapies [6].”
BackgroundSelf-citation
Flagged first, then by citations
Research works printed but not linked: 1
- A.R. MacLeod, H.S. Gill, Generating preclinical evidence for MHRA - an in silico clinical trial examining the safety of a novel device for knee arthritis treatment,” presented at the Bath Biomechanics Symposium 2019, Bath, 2019.
Cited by
Sorted by citations
Sabe, V. T., Ntombela, T., Jhamba, L. A., Maguire, G. E. M., Govender, T., Naicker, T., & Kruger, H. G. (2021) · European Journal of Medicinal Chemistry · doi:10.1016/j.ejmech.2021.113705 · Cited by 665
Musuamba, F. T., Rusten, I. S., Lesage, R., Russo, G. L., Bursi, R., Emili, L., Wangorsch, G., Manolis, E., Karlsson, K. E., Kulesza, A. J., et al. (2021) · CPT Pharmacometrics & Systems Pharmacology · doi:10.1002/psp4.12669 · Cited by 175
Self-citationArtificial intelligence and machine learning applications in biopharmaceutical manufacturing
Rathore, A. S., Nikita, S., Thakur, G., & Mishra, S. (2022) · Trends in biotechnology · doi:10.1016/j.tibtech.2022.08.007 · Cited by 174
Finite Element Analysis of Fracture Fixation
Lewis, G. S., Mischler, D., Wee, H., Reid, J. S., & Varga, P. (2021) · Current Osteoporosis Reports · doi:10.1007/s11914-021-00690-y · Cited by 152
Emerging technologies and their impact on regulatory science
Anklam, E., Bahl, M. I., Ball, R., Beger, R. D., Cohen, J. E., Fitzpatrick, S., Girard, P., Halamoda‐Kenzaoui, B., Hinton, D., Hirose, A., et al. (2021) · Experimental Biology and Medicine · doi:10.1177/15353702211052280 · Cited by 134
Metadata
- Authors
- Viceconti, M., Pappalardo, F., Rodriguez, B., Horner, M., Bischoff, J., & Musuamba Tshinanu, F.
- Affiliations
- Istituto Ortopedico Rizzoli (IT)
- University of Catania (IT)
- Corresponding author
- Viceconti, Marco
- Venue
- Methods
- Publisher
- Elsevier BV
- Published
- 2021-01
- Volume
- 185
- Pages
- 120-127
- ISSN
- 1046-2023 (ISSN-L)
- PMC
- PMC7883933
- OpenAlex
- W3001465925
- Cited by
- 361
- References
- 48
- Citations by year
- 2026: 85 · 2025: 71 · 2024: 72 · 2023: 48 · 2022: 33 · 2021: 44
- FWCI
- 31.72
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Methods (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Archivio istituzionale della ricerca (Alma Mater Studiorum Università di Bologna) (submittedVersion, cc-by-nc-nd)
- INFM-OAR (INFN Catania) (submittedVersion, cc-by)
- Funders
- Wellcome Trust
- Topics
- Health Sciences › Veterinary › Small Animals › Animal testing and alternatives
- Life Sciences › Pharmacology, Toxicology and Pharmaceutics › Drug Discovery › Computational Drug Discovery Methods
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Keywords
- verification and validation, uncertainty quantification, biophysical model, regulatory evaluation, drug approval, context of use, risk analysis, acceptability threshold, prediction model, regulatory qualification
- MeSH terms
- Machine Learning, Computer Simulation, Models, Theoretical, Uncertainty
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
