The DOI matches its Crossref record; 72 of the 72 references it cites were found in a registry or an index; 61 of the 99 sources its claims cite were checked for support.
Credibility assessment of in silico clinical trials for medical devices
Open accessAbstract
In silico clinical trials (ISCTs) are an emerging method in modeling and simulation where medical interventions are evaluated using computational models of patients. ISCTs have the potential to provide cost-effective, time-efficient, and ethically favorable alternatives for evaluating the safety and effectiveness of medical devices. However, ensuring the credibility of ISCT results is a significant challenge. This paper aims to identify unique considerations for assessing the credibility of ISCTs and proposes an ISCT credibility assessment workflow based on recently published model assessment frameworks. First, we review various ISCTs described in the literature, carefully selected to showcase the range of methodological options available. These studies cover a wide variety of devices, reasons for conducting ISCTs, patient model generation approaches including subject-specific versus ‘synthetic’ virtual patients, complexity levels of devices and patient models, incorporation of clinician or clinical outcome models, and methods for integrating ISCT results with real-world clinical trials. We next discuss how verification, validation, and uncertainty quantification apply to ISCTs, considering the range of ISCT approaches identified. Based on our analysis, we then present a hierarchical workflow for assessing ISCT credibility, using a general credibility assessment framework recently published by the FDA’s Center for Devices and Radiological Health. Overall, this work aims to promote standardization in ISCTs and contribute to the wider adoption and acceptance of ISCTs as a reliable tool for evaluating medical devices.
Overview
CoChat Summary
TL;DR
This paper reviews eight medical-device in silico clinical trials (ISCTs) and proposes a hierarchical workflow for assessing ISCT credibility based on FDA guidance and ASME V&V40. It argues that ISCT credibility requires sub-model-level verification, validation, and uncertainty quantification, plus population-level validation of the virtual cohort.
Key findings
- ISCTs are composed of multiple sub-models—baseline patient model, device model, virtual cohort, and optionally clinical outcome, clinician, emulator, and other system models—so credibility assessment should be hierarchical rather than only top-level.
- The reviewed examples span subject-specific, synthetic, and mixed virtual cohorts: MR-induced heating used 15 subject-specific models from 12 subjects; VICTRE used 2,986 synthetic patients; FD-PASS used 164 virtual patients from 82 anatomies; the artificial pancreas example used 300 synthetic patients; whole-heart CRT models used 24 subject-specific models altered to heart failure; automata-based ICD models used >11,000 virtual patients; and Micra involved 50 million simulations total.
- Validation can occur at sub-model level (device, patient, coupled device-patient, clinical outcome, clinician) and at cohort level, either individual-level with paired data or population-level without paired data; population-level validation alone risks “right answer for the wrong reason.”
- The proposed workflow aligns with FDA credibility categories and adds credibility factors/gradations for population-level validation of the virtual cohort, including clinical dataset size, subject range, output comparison rigor, and agreement.
Methods
- Narrative review of eight selected ISCTs for medical devices, chosen to showcase methodological range rather than to comprehensively survey all ISCTs.
- Qualitative synthesis of how verification, validation, and uncertainty quantification apply to ISCTs, including sub-model and cohort validation strategies.
- Proposal of a hierarchical ISCT credibility assessment workflow based on FDA guidance “Assessing the Credibility of Computational Modeling and Simulation in Medical Device Submissions” and ASME V&V40-2018, illustrated with a hypothetical whole-heart/pacemaker-lead ISCT.
Limitations
- The paper states the workflow may be perceived as overly burdensome, may require case-specific adjustments, and such adjustments could introduce subjectivity.
- The paper states its scope is first-principles models; data-driven models are not harmonized with first-principles credibility assessment, and ISCT execution is outside scope.
- The paper states there is no widely accepted community definition of an ISCT, and the literature review is not comprehensive; the proposed workflow also assumes specific features such as a physics-based device model, first-principles patient model, no emulator or explicit clinician model, synthetic cohort, and no Bayesian ISCT/CT integration.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures




Verification
Claims the paper makes
45 Valid · 9 Not valid · 24 Not checked · 105 Own results and interpretations
Every source the paper cites for this claim was checked and supports it.
In the paper
“The flow diverter performance assessment (FD-PASS) study [16] was an in silico clinical trial that predicted post-treatment flow reduction in 164 virtual patients using computational fluid dynamics (CFD).”
§ Credibility assessment of in silico clinical trials for medical devices > 2. In silico clinical trials for medical devices > 2.3. Example ISCTs for medical devices
What the sources say
- Supportedcites [16] A Sarrami-Foroushani, 2021, In-silico trial of intracranial flow diverters replicates and expands insights from conventional clinical trials, Nature communications source
The Introduction’s Fig. 1 workflow describes 82 virtual anatomies modeled under normotensive and hypertensive conditions (N = 164), with the 3-D Navier–Stokes equations solved to measure post-treatment aneurysm mean velocity reduction. The Introduction also identifies FD-PASS as an in-silico trial with post-treatment velocity reduction as its primary endpoint. (LLM, conf=0.93)
plainstates a causen = 164- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Studies where multiple modeling groups attempt to solve the same pre-specified problem have been useful in various fields for code verification and, more generally, for scientific reproducibility [48–50].”
§ Credibility assessment of in silico clinical trials for medical devices > 3. Verification, validation and uncertainty quantification for ISCTs > 3.1. Verification
What the sources say
- Partly supportedcites [48] K Valen-Sendstad, 2018, Real-world variability in the prediction of intracranial aneurysm wall shear stress: the 2015 international aneurysm CFD challenge, Cardiovascular engineering and technology sourceExperimental — accuracy not yet measured
The Methods (Response to the Challenge; page not provided) describe 26 modeling teams submitting CFD results, and the Conclusions report variability that impedes combining results across groups. This supports the study as an example of a multi-group reproducibility challenge, but our copy does not establish that such studies have been useful across various fields or for code verification. (LLM, conf=0.84)
- Supportedcites [49] SA Niederer, 2011, Verification of cardiac tissue electrophysiology simulators using an N-version benchmark, Phil. Trans. R. Soc. A source
Section 2 describes multi-code comparisons of common problems in earth science, astronomy, meteorology and mantle simulation as code-verification strategies. It says the shared-benchmark approach also demonstrates that a benchmark is reproducible; the Benchmark utility section describes this study as confirming the reproducibility of multi-scale simulations. (LLM, conf=0.91)
- Partly supportedcites [50] I Sechopoulos, 2015, Monte Carlo reference data sets for imaging research: Executive summary of the report of AAPM Research Committee Task Group 195, Medical physics sourceExperimental — accuracy not yet measured
The Abstract and Section 2.A describe four Monte Carlo packages applied to the same specified imaging simulations, and the Discussion explains how comparing their results helps benchmark and validate simulation codes. The paper supports that example, but does not establish usefulness across various fields or for scientific reproducibility more generally. (LLM, conf=0.85)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“The results of the ISCT compared favorably with the results of an existing clinical trial that was used as a reference”
§ Credibility assessment of in silico clinical trials for medical devices > 2. In silico clinical trials for medical devices > 2.3. Example ISCTs for medical devices
What the sources say
- Supportedcites [32] A Badano, 2018, Evaluation of digital breast tomosynthesis as replacement of full-field digital mammography using an in silico imaging trial, JAMA network open source
The Results state that VICTRE’s differential performance was consistent with the comparative trial’s aggregate results and its stronger performance for detecting masses than microcalcifications. The Discussion likewise says the overall results were consistent with the comparative trial, supporting the claim that the in silico trial compared favorably with its clinical reference. (LLM, conf=0.93)
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“the Virtual Imaging Clinical Trial for Regulatory Evaluation (VICTRE) project [32] explored the possibility of using an ISCT to compare the performance of FFDM and DBT.”
§ Credibility assessment of in silico clinical trials for medical devices > 2. In silico clinical trials for medical devices > 2.3. Example ISCTs for medical devices
What the sources say
- Supportedcites [32] A Badano, 2018, Evaluation of digital breast tomosynthesis as replacement of full-field digital mammography using an in silico imaging trial, JAMA network open source
The Discussion describes VICTRE as an in silico trial comparing digital mammography (DM) with digital breast tomosynthesis (DBT), and reports their differential performance. The paper’s title identifies the mammography modality as full-field digital mammography. Page numbers were not available in our copy. (LLM, conf=0.94)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“First, we noted that with PSMs it is common to perform calculation verification studies such as grid refinement studies using a single, assumed representative, patient.”
§ Credibility assessment of in silico clinical trials for medical devices > 3. Verification, validation and uncertainty quantification for ISCTs > 3.1. Verification
What the sources say
- Supportedcites [9] SK Galappaththige, 2022, Credibility Assessment of Patient-Specific Computational Modeling using Patient-Specific Cardiac Modeling as an Exemplar, PLOS Computational Biology source
Section 2.2 says that in cardiac patient-specific models, often either no verification is performed or a single mesh-resolution study is used to justify the resolution for all patients. This supports the claim that, when such a study is performed, it commonly uses one patient as representative of the others. (LLM, conf=0.92)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
33 more claims · 6 with their cited sources checked
Create a free account to see all 183 claims and to verify the 2 references still open.
- Probably not a substantial claim. 126
- It may state two separate claims as one. 49
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Credibility assessment of in silico clinical trials for medical devices > 2. In silico clinical trials for medical devices > 2.3. Example ISCTs for medical devices
“10K simulated trials of 5K virtual patients each, 50M simulations total”
§ Credibility assessment of in silico clinical trials for medical devices > 2. In silico clinical trials for medical devices > 2.3. Example ISCTs for medical devices
“The study involved 5,000 simulated trials, each comprised of 10,000 virtual patients, for a total of 50 million simulations.”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Credibility assessment of in silico clinical trials for medical devices > 3. Verification, validation and uncertainty quantification for ISCTs > 3.2. Validation
“enabling validation of both the virtual patient in isolation and the coupled device-patient model (Fig 2d; see [56] for an example in stent-graft deployment)”
§ Credibility assessment of in silico clinical trials for medical devices > References
“56. O’Hara T, Virag L, Varro A, Rudy Y, Simulation of the undiseased human cardiac ventricular action potential: model formulation and experimental validation.”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Credibility assessment of in silico clinical trials for medical devices > 3. Verification, validation and uncertainty quantification for ISCTs > 3.2. Validation
“If subject-specific virtual patients are generated (whether for cohort generation approaches A, B or C in Fig 1), validation may be possible against other data collected from the same subjects–see Fig 2b.”
§ Credibility assessment of in silico clinical trials for medical devices > 3. Verification, validation and uncertainty quantification for ISCTs > 3.2. Validation
“| Validate patient model | Fig 2b | Patient model using nominal or population-averaged parameter values | Population-averaged clinical data | Cardiac modeling [56] | 4 |”
Citations
Cites
An Introduction to Statistical Learning
James, G., Witten, D., Hastie, T., & Tibshirani, R. (2013) · Springer Texts in Statistics · doi:10.1007/978-1-4614-7138-7 · Cited by 12,106
Physics-informed machine learning
Karniadakis, G. E., Kevrekidis, I. G., Lu, L., Perdikaris, P., Wang, S., & Yang, L. (2021) · Nature Reviews Physics · doi:10.1038/s42254-021-00314-5 · Cited by 8,718
Understanding bland altman analysis
Giavarina, D. (2015) · Biochemia Medica · doi:10.11613/bm.2015.015 · Cited by 4,706
Verification and Validation in Scientific Computing
Oberkampf, W. L., & Roy, C. J. (2010) · doi:10.1017/cbo9780511760396 · Cited by 1,556
Receiver-operating characteristic analysis for evaluating diagnostic tests and predictive models
Zou, K. H., O’Malley, A. J., & Mauri, L. (2007) · Circulation · doi:10.1161/circulationaha.105.594929 · Cited by 1,473
O'Hara, T., Virág, L., Varró, A., & Rudy, Y. (2011) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1002061 · Cited by 1,196
The ‘Digital Twin’ to enable the vision of precision cardiology
Corral-Acero, J., Margara, F., Marciniak, M., Rodero, C., Loncaric, F., Feng, Y., Gilbert, A., Fernandes, J. F., Bukhari, H. A., Wajdan, A., et al. (2020) · European Heart Journal · doi:10.1093/eurheartj/ehaa159 · Cited by 932
Self-citationExternal validation of prognostic models: what, why, how, when and where?
Ramspek, C. L., Jager, K. J., Dekker, F. W., Zoccali, C., & van Diepen, M. (2021) · Clinical Kidney Journal · doi:10.1093/ckj/sfaa188 · Cited by 887
The UVA/PADOVA Type 1 Diabetes Simulator: New Features
Man, C. D., Micheletto, F., Lv, D., Breton, M., Kovatchev, B., & Cobelli, C. (2014) · Journal of Diabetes Science and Technology · doi:10.1177/1932296813514502 · Cited by 791 · cited 1×
“Another example is compartmental glucose-insulin models, which have been used to generate virtual cohorts to evaluate automated insulin pumps [38,39] (more details in Table 1).”
In Silico Preclinical Trials: A Proof of Concept in Closed-Loop Control of Type 1 Diabetes
Kovatchev, B. P., Breton, M., Dalla Man, C., & Cobelli, C. (2009) · Journal of Diabetes Science and Technology · doi:10.1177/193229680900300106 · Cited by 760
Lorenz, C., Walker, E., Morgan, V., Klein, S., & Graham, T. (1999) · Journal of Cardiovascular Magnetic Resonance · doi:10.3109/10976649909080829 · Cited by 717 · cited 1×
“For example, women are known to have on average smaller hearts and have differences in electrophysiology [69, 70], and this relationship between heart size and tissue properties may not be explicitly represented in a model.”
Gosselin, M.-C., Neufeld, E., Moser, H., Huber, E., Farcito, S., Gerber, L., Jedensjö, M., Hilber, I., Gennaro, F. D., Lloyd, B., et al. (2014) · Physics in Medicine and Biology · doi:10.1088/0031-9155/59/18/5287 · Cited by 520
Surrogates: Gaussian Process Modeling, Design, and Optimization for the Applied Sciences
Gramacy, R. B. (2020) · doi:10.1201/9780367815493 · Cited by 477
Kudela, J., & Matousek, R. (2022) · Soft Computing · doi:10.1007/s00500-022-07362-8 · Cited by 370
Verification of cardiac tissue electrophysiology simulators using an N -version benchmark
Niederer, S. A., Kerfoot, E., Benson, A. P., Bernabeu, M. O., Bernus, O., Bradley, C., Cherry, E. M., Clayton, R., Fenton, F. H., Garny, A., et al. (2011) · Philosophical Transactions of the Royal Society A Mathematical Physical and Engineering Sciences · doi:10.1098/rsta.2011.0139 · Cited by 275
Self-citationMathematical models of bioheat transfer
Charny, C. K. (1992) · Advances in heat transfer · doi:10.1016/s0065-2717(08)70344-7 · Cited by 241
Badano, A., Graff, C. G., Badal, A., Sharma, D., Zeng, R., Samuelson, F. W., Glick, S. J., & Myers, K. J. (2018) · JAMA Network Open · doi:10.1001/jamanetworkopen.2018.5474 · Cited by 235
Self-citationGaussian process emulation of dynamic computer codes
Conti, S., Gosling, J. P., Oakley, J. E., & O'Hagan, A. (2009) · Biometrika · doi:10.1093/biomet/asp028 · Cited by 195 · cited 1×
“Emulators are often statistical models, for example, Gaussian process emulators [23, 24], although they could also utilize lower-fidelity physics-based models [25].”
Cyber–Physical Modeling of Implantable Cardiac Medical Devices
Jiang, Z., Pajic, M., & Mangharam, R. (2012) · Proceedings of the IEEE · doi:10.1109/jproc.2011.2161241 · Cited by 172
Length-dependent tension in the failing heart and the efficacy of cardiac resynchronization therapy
Niederer, S., Plank, G., Chinchapatnam, P. P., Ginks, M., Lamata, P., Rhode, K., Rinaldi, C. A., Razavi, R., & Smith, N. P. (2010) · Cardiovascular Research · doi:10.1093/cvr/cvq318 · Cited by 159
Self-citationCreation and application of virtual patient cohorts of heart models
Niederer, S., Aboelkassem, Y., Cantwell, C. D., Corrado, C., Coveney, S., Cherry, E. M., Delhaas, T., Fenton, F. H., Panfilov, A. V., Pathmanathan, P., et al. (2020) · Philosophical Transactions of the Royal Society A Mathematical Physical and Engineering Sciences · doi:10.1098/rsta.2019.0558 · Cited by 131 · cited 1×
“The baseline patient model has also been referred to as the template model [18].”
Self-citationValen-Sendstad, K., Bergersen, A. W., Shimogonya, Y., Goubergrits, L., Bruening, J., Pallares, J., Cito, S., Piskin, S., Pekkan, K., Geers, A. J., et al. (2018) · Cardiovascular Engineering and Technology · doi:10.1007/s13239-018-00374-2 · Cited by 128 · cited 2×
“Studies where multiple modeling groups attempt to solve the same pre-specified problem have been useful in various fields for code verification and, more generally, for scientific reproducibility [48–50].”
Erdemir, A., Mulugeta, L., Ku, J. P., Drach, A., Horner, M., Morrison, T. M., Peng, G. C. Y., Vadigepalli, R., Lytton, W. W., & Myers, J. G. (2020) · Journal of Translational Medicine · doi:10.1186/s12967-020-02540-4 · Cited by 118 · cited 1×
“The medical devices modeling community has developed frameworks and methods to assess credibility of computational models for medical devices, including the ASME V&V40 2018 Standard, FDA Guidance [2], and various supporting documents [3–7].”
Physical validation of a patient-specific contact finite element model of the ankle
Anderson, D. D., Goldsworthy, J. K., Li, W., et al. (2007) · Journal of Biomechanics · doi:10.1016/j.jbiomech.2007.01.024 · Cited by 116
Strocchi, M., Augustin, C. M., Gsell, M. A. F., Karabelas, E., Neic, A., Gillette, K., Razeghi, O., Prassl, A. J., Vigmond, E. J., Behar, J. M., et al. (2020) · PLoS ONE · doi:10.1371/journal.pone.0235145 · Cited by 112
Self-citationClosed-Loop Control of Anesthesia
Dumont, G. A., & Ansermino, J. M. (2013) · Anesthesia & Analgesia · doi:10.1213/ane.0b013e3182973687 · Cited by 110
Sechopoulos, I., Ali, E. S. M., Badal, A., Badano, A., Boone, J. M., Kyprianou, I. S., Mainegra‐Hing, E., McMillan, K. L., McNitt‐Gray, M. F., Rogers, D. W. O., et al. (2015) · Medical Physics · doi:10.1118/1.4928676 · Cited by 109
Self-citationKainz, W., Neufeld, E., Bolch, W. E., Graff, C. G., Kim, C. H., Kuster, N., Lloyd, B., Morrison, T., Segars, P., Yeom, Y. S., et al. (2019) · IEEE Transactions on Radiation and Plasma Medical Sciences · doi:10.1109/trpms.2018.2883437 · Cited by 109 · cited 1×
“A well-established method addressing this safety question is computational electromagnetic modeling [27,28].”
Lee, J. H., Rygg, A. D., Kolahdouz, E. M., Rossi, S., Retta, S. M., Duraiswamy, N., Scotten, L. N., Craven, B. A., & Griffith, B. E. (2020) · Annals of Biomedical Engineering · doi:10.1007/s10439-020-02466-4 · Cited by 108
Self-citationPatient-specific numerical simulation of stent-graft deployment: Validation on three clinical cases
Perrin, D., Badel, P., Orgéas, L., et al. (2015) · Journal of Biomechanics · doi:10.1016/j.jbiomech.2015.04.031 · Cited by 101
Sarrami-Foroushani, A., Lassila, T., MacRaild, M., Asquith, J., Roes, K. C. B., Byrne, J. V., & Frangi, A. F. (2021) · Nature Communications · doi:10.1038/s41467-021-23998-w · Cited by 83
Multiple aneurysms anatomy challenge 2018 (MATCH): phase I: segmentation
Berg, P., Voß, S., Saalfeld, S., Janiga, G., Bergersen, A. W., Valen-Sendstad, K., Bruening, J., Goubergrits, L., Spuler, A., Cancelliere, N. M., et al. (2018) · Cardiovascular Engineering and Technology · doi:10.1007/s13239-018-00376-0 · Cited by 79 · cited 1×
“For such PSMs, it is important to confirm that outputs of the model are not impacted by inter-or intra-user variability [9,48,51,52].”
Strocchi, M., Lee, A. W. C., Neic, A., Bouyssier, J., Gillette, K., Plank, G., Elliott, M. K., Gould, J., Behar, J. M., Sidhu, B., et al. (2020) · Heart Rhythm · doi:10.1016/j.hrthm.2020.06.028 · Cited by 73
Applicability Analysis of Validation Evidence for Biomedical Computational Models
Pathmanathan, P., Gray, R. A., Romero, V. J., & Morrison, T. M. (2017) · Journal of Verification Validation and Uncertainty Quantification · doi:10.1115/1.4037671 · Cited by 68
Self-citationMorrison, T. M., Hariharan, P., Funkhouser, C. M., Afshari, P., Goodin, M., & Horner, M. (2019) · ASAIO Journal · doi:10.1097/mat.0000000000000996 · Cited by 68
Credibility of In Silico Trial Technologies—A Theoretical Framing
Viceconti, M., Juarez, M. A., Curreli, C., Pennisi, M., Russo, G., & Pappalardo, F. (2020) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2019.2949888 · Cited by 67
Validation and Trustworthiness of Multiscale Models of Cardiac Electrophysiology
Pathmanathan, P., & Gray, R. A. (2018) · Frontiers in Physiology · doi:10.3389/fphys.2018.00106 · Cited by 63
Self-citationHaddad, T., Himes, A., Thompson, L., Irony, T., Nair, R., & on Behalf of MDIC Computer Modeling and Simulation Working Group Participants (2017) · Journal of Biopharmaceutical Statistics · doi:10.1080/10543406.2017.1300907 · Cited by 62 · cited 7×
“The question of how to augment a real-world clinical trial with results from an ISCT is addressed in [26].”
InfluentialEmendi, M., Sturla, F., Ghosh, R. P., Bianchi, M., Piatti, F., Pluchinotta, F. R., Giese, D., Lombardi, M., Redaelli, A. C. L., & Bluestein, D. (2020) · Annals of Biomedical Engineering · doi:10.1007/s10439-020-02571-4 · Cited by 59
Basic Framework and Main Methods of Uncertainty Quantification
Zhang, J., Yin, J., & Wang, R. (2020) · Mathematical Problems in Engineering · doi:10.1155/2020/6068203 · Cited by 52 · cited 1×
“Methods for performing such a bottom-up analysis are available in the UQ literature [72–74].”
Corrado, C., Williams, S., Karim, R., Plank, G., O’Neill, M., & Niederer, S. (2018) · Medical Image Analysis · doi:10.1016/j.media.2018.04.005 · Cited by 50 · cited 1×
“As an example, [64] constructs seven patient-specific heart (left atrium) models using electro-anatomical data.”
Galappaththige, S., Gray, R. A., Costa, C. M., Niederer, S., & Pathmanathan, P. (2022) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1010541 · Cited by 45
Parvinian, B., Pathmanathan, P., Daluwatte, C., Yaghouby, F., Gray, R. A., Weininger, S., Morrison, T. M., & Scully, C. G. (2019) · Frontiers in Physiology · doi:10.3389/fphys.2019.00220 · Cited by 44
Self-citationJin, X., Bighamian, R., & Hahn, J. (2018) · IEEE Transactions on Biomedical Engineering · doi:10.1109/tbme.2018.2880927 · Cited by 40 · cited 1×
“Examples of PCLC devices include autonomous mechanical ventilators that constantly adjust a patient’s end-tidal oxygen [33], automated anesthetic devices [34], automated fluid resuscitation devices [35], or automated insulin pumps that measure blood glucose levels and deliver insulin accordingly.”
Self-citationUncertainty quantification of simulated biomechanical stimuli in coronary artery bypass grafts
Tran, J. S., Schiavazzi, D. E., Kahn, A. M., & Marsden, A. L. (2019) · Computer Methods in Applied Mechanics and Engineering · doi:10.1016/j.cma.2018.10.024 · Cited by 37
Hariharan, P., D’Souza, G. A., Horner, M., Morrison, T. M., Malinauskas, R. A., & Myers, M. R. (2017) · PLoS ONE · doi:10.1371/journal.pone.0178749 · Cited by 35
Effect of scar and His–Purkinje and myocardium conduction on response to conduction system pacing
Strocchi, M., Gillette, K., Neic, A., Elliott, M. K., Wijesuriya, N., Mehta, V., Vigmond, E. J., Plank, G., Rinaldi, C. A., & Niederer, S. A. (2023) · Journal of Cardiovascular Electrophysiology · doi:10.1111/jce.15847 · Cited by 35 · cited 3×
“To validate the cohort, the authors compared cohort predictions for different pathology presentations and responses to therapy to summary statistics from independent clinical studies [45].”
InfluentialIn silico clinical trials in the orthopedic device industry: from fantasy to reality?
Favre, P., Maquer, G., Henderson, A., Hertig, D., Ciric, D., & Bischoff, J. E. (2021) · Annals of Biomedical Engineering · doi:10.1007/s10439-021-02787-y · Cited by 32 · cited 1×
“…related work, including a theoretical framing for credibility of in silico studies [10], a discussion of limitations of ASME V&V40 2018 for ISCTs [11], a framework for executing an ISCT using VVUQ concepts [12], and a proposed alteration to ASME V&V40 2018 for ISCTs, including modification of…”
Evaluation of Fluid Resuscitation Control Algorithms via a Hardware-in-the-Loop Test Bed
Mirinejad, H., Parvinian, B., Ricks, M., Zhang, Y., Weininger, S., Hahn, J., & Scully, C. G. (2019) · IEEE Transactions on Biomedical Engineering · doi:10.1109/tbme.2019.2915526 · Cited by 30
Solís-Lemus, J. A., Baptiste, T. M. G., Barrows, R. K., Sillett, C., Gharaviri, A., Raffaele, G., Razeghi, O., Strocchi, M., Sim, I., Kotadia, I., et al. (2023) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2023.107009 · Cited by 29
Self-citationGao, H., Zhu, X., & Wang, J.-X. (2020) · Computer Methods in Applied Mechanics and Engineering · doi:10.1016/j.cma.2020.113047 · Cited by 27
Antonini, L., Mandelli, L., Berti, F., Pennati, G., & Petrini, L. (2021) · Journal of the mechanical behavior of biomedical materials/Journal of mechanical behavior of biomedical materials · doi:10.1016/j.jmbbm.2021.104644 · Cited by 27
On the model validation of active implantable medical device for MRI safety assessment
Wang, Z., Zheng, J., Wang, Y., Kainz, W., & Chen, J. (2020) · IEEE Transactions on Microwave Theory and Techniques · doi:10.1109/tmtt.2019.2957766 · Cited by 22 · cited 1×
“For example, for MR-induced heating (Section 2.3.1), a model of the power absorption of a new device can be tested by comparing against experiments using the same device in a saline-gel phantom [53].”
InfluentialBadano, A. G., Lago, M. A., Sizikova, E., Delfino, J. G., Guan, S., Anastasio, M. A., & Sahiner, B. (2023) · Progress in Biomedical Engineering · doi:10.1088/2516-1091/ad04c0 · Cited by 21
Cito, S., Geers, A. J., Arroyo, M. P., Palero, V. R., Pallarés, J., Vernet, A., Blasco, J., San Román, L., Fu, W., Qiao, A., et al. (2015) · Annals of Biomedical Engineering · doi:10.1007/s10439-014-1082-9 · Cited by 20 · cited 1×
“For example, [60] validates an aneurysm flow model against flow measurements in a corresponding glass model of the aneurysm and surrounding vasculature.”
A risk and credibility framework for in silico clinical trials of medical devices
Bischoff, J. E., Dharia, M. A., & Favre, P. (2023) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2023.107813 · Cited by 20 · cited 1×
“…studies [10], a discussion of limitations of ASME V&V40 2018 for ISCTs [11], a framework for executing an ISCT using VVUQ concepts [12], and a proposed alteration to ASME V&V40 2018 for ISCTs, including modification of the risk assessment stage which is outside the scope of the present paper [13].”
Uncertainty quantification patterns for multiscale models
Ye, D., Veen, L., Nikishova, A., Lakhlili, J., Edeling, W., Luk, O. O., Krzhizhanovskaya, V. V., & Hoekstra, A. G. (2021) · Philosophical Transactions of the Royal Society A Mathematical Physical and Engineering Sciences · doi:10.1098/rsta.2020.0072 · Cited by 19
Gender-based differences in cardiac disease
Yang, P.-C., & Clancy, C. E. (2011) · Journal of Biomedical Research · doi:10.1016/s1674-8301(11)60010-9 · Cited by 16
In-silico pre-clinical trials for implantable cardioverter defibrillators
Jiang, Z., Abbas, H., Jang, K. J., Beccani, M., Liang, J., Dixit, S., & Mangharam, R. (2016) · 2016 38th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) · doi:10.1109/embc.2016.7590667 · Cited by 16
Strocchi, M., Gillette, K., Neic, A., Elliott, M. K., Wijesuriya, N., Mehta, V., Vigmond, E. J., Plank, G., Rinaldi, C. A., & Niederer, S. A. (2022) · Frontiers in Physiology · doi:10.3389/fphys.2022.1011566 · Cited by 16
Self-citationVirtual Patient Generation using Physiological Models through a Compressed Latent Parameterization
Tivay, A., Kramer, G. C., & Hahn, J.-O. (2020) · 2020 American Control Conference (ACC) · doi:10.23919/acc45564.2020.9147298 · Cited by 12
Bodner, J., & Kaul, V. (2021) · ASME Verification and Validation Symposium (VVS) · doi:10.1115/vvs2021-65094 · Cited by 11
Self-citationLi, Y., Wang, J., Zhang, W., et al. (2021) · PubMed · doi:10.3969/j.issn.1671-7104.2021.01.019 · Cited by 10
Development and validation of a mathematical model of heart rate response to fluid perturbation
Kanal, V., Pathmanathan, P., Hahn, J.-O., Kramer, G., Scully, C., & Bighamian, R. (2022) · Scientific Reports · doi:10.1038/s41598-022-25891-y · Cited by 10
Self-citationStrocchi, M., Wijesuriya, N., Elliott, M. K., Gillette, K., Neic, A., Mehta, V., Vigmond, E. J., Plank, G., Rinaldi, C. A., & Niederer, S. A. (2022) · Frontiers in Physiology · doi:10.3389/fphys.2022.1049214 · Cited by 8 · cited 1×
“These models were used in [41–43] to study the efficacy of different cardiac resynchronization therapy (CRT) pacing protocols for heart failure patients with left bundle branch block (LBBB).”
Bodner, J., & Kaul, V. (2022) · Journal of Verification Validation and Uncertainty Quantification · doi:10.1115/1.4053565 · Cited by 7
Self-citationRobustness evaluation of computer-aided clinical trials for medical devices
Jang, K. J., Pant, Y. V., Zhang, B., Weimer, J., & Mangharam, R. (2019) · Proceedings of the 10th ACM/IEEE International Conference on Cyber-Physical Systems · doi:10.1145/3302509.3311058 · Cited by 6 · cited 1×
“A related work, [47], uses the same model within a Bayesian statistical framework for integrating the ISCT with a concurrent CT, using methods related to Section 2.3.6, and additionally presents a method for quantifying the robustness of the ISCT to underlying modeling assumptions.”
InfluentialUse of Real-World Data for Enhancing Model Credibility: Applications to Medical Device Development
Briant, P., Bischoff, J. E., Dharia, M. A., Le Navéaux, F., Li, X., Kulkarni, S., Levine, D., Ramos, D. O., & Afshari, P. (2022) · Journal of Medical Devices · doi:10.1115/1.4053888 · Cited by 6
Guillem, M. S., Pujol‐López, M., Sanchez‐Arciniegas, J., & Mont, L. (2023) · Journal of Cardiovascular Electrophysiology · doi:10.1111/jce.15871 · Cited by 4
The role of artificial intelligence within in silico medicine
Geris, L., Rousseau, C., Noailly, J., et al. (2022) · HAL (Le Centre pour la Communication Scientifique Directe) · doi:10.5281/zenodo.17511399 · Cited by 2
Solís-Lemus, J. A., Baptiste, T. M. G., Barrows, R. K., et al. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2301.06998 · Cited by 1
Erratum to “On the Model Validation of Active Implantable Medical Device for MRI Safety Assessment”
Wang, Z., Zheng, J., Wang, Y., Kainz, W., & Chen, J. (2020) · IEEE Transactions on Microwave Theory and Techniques · doi:10.1109/tmtt.2020.2978595 · Cited by 1
Cited by
The future of in silico trials and digital twins in medicine
Samei, E. (2025) · PNAS Nexus · doi:10.1093/pnasnexus/pgaf123 · Cited by 39
Aycock, K. I., Battisti, T., Peterson, A., Jiang, Y. D., Kreuzer, S. M., Capelli, C., Pant, S., Pathmanathan, P., Hoganson, D. M., Levine, S. M., et al. (2024) · Frontiers in Medicine · doi:10.3389/fmed.2024.1433372 · Cited by 40
Self-citationAlharthi, S. (2025) · Saudi Pharmaceutical Journal · doi:10.1007/s44446-025-00055-x · Cited by 43
“…a patient, organ, or biological system that integrates multidimensional, patient-specific data to support clinical decision-making (Drummond & Gonsard 2024), while ISTs refer to structured and comprehensive computational simulations conducted using or in conjunction with DTs (Table 2) (Samei 2025).”
MASLD, MASH, and the CKM Spectrum
Zannad, F. M., Khan, M. S., Bansal, N., Böhm, M., Francque, S., Girerd, N., Januzzi, J. L., Miller, V., Piché, M., Ratziu, V., et al. (2026) · Journal of the American College of Cardiology · doi:10.1016/j.jacc.2025.12.015 · Cited by 40
Toward widespread use of virtual trials in medical imaging innovation and regulatory science
Abadi, E., Barufaldi, B., Lago, M. A., Badal, A., Mello-Thoms, C. R., Bottenus, N. B., Wangerin, K. A., Goldburgh, M. M., Tarbox, L., Beaucage‐Gauvreau, E., et al. (2024) · Medical Physics · doi:10.1002/mp.17442 · Cited by 19
Self-citation
Metadata
- Authors
- Pathmanathan, P., Aycock, K., Badal, A., Bighamian, R., Bodner, J., Craven, B. A., & Niederer, S.
- Affiliations
- United States Food and Drug Administration (US)
- Medtronic (United States) (US)
- The Alan Turing Institute (GB)
- Corresponding author
- Pathmanathan, Pras
- ORCID
- Venue
- PLOS Computational Biology
- Publisher
- Public Library of Science (PLoS)
- Published
- 2024-08-08
- Volume
- 20
- Issue
- 8
- Pages
- e1012289
- ISSN
- 1553-734X (ISSN-L)
- 1553-7358
- PMC
- PMC11309390
- OpenAlex
- W4401418803
- Cited by
- 62
- References
- 70
- Citations by year
- 2026: 30 · 2025: 27 · 2024: 5
- FWCI
- 21.28
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- public-domain
- creativecommons.org/publicdomain/zero/1.0/ (vor)
- Locations
- PLoS Computational Biology (publishedVersion, public-domain)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Social Sciences › Education › Simulation-Based Education in Healthcare
- Keywords
- in silico clinical trials, medical devices, credibility assessment, model verification and validation, uncertainty quantification, patient modeling, virtual patients, subject-specific modeling, standardization
- MeSH terms
- Clinical Trials as Topic, Computer Simulation, Equipment and Supplies, Reproducibility of Results
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.