Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record; 72 of the 72 references it cites were found in a registry or an index; 61 of the 99 sources its claims cite were checked for support.

Credibility assessment of in silico clinical trials for medical devices

Open access

Abstract

In silico clinical trials (ISCTs) are an emerging method in modeling and simulation where medical interventions are evaluated using computational models of patients. ISCTs have the potential to provide cost-effective, time-efficient, and ethically favorable alternatives for evaluating the safety and effectiveness of medical devices. However, ensuring the credibility of ISCT results is a significant challenge. This paper aims to identify unique considerations for assessing the credibility of ISCTs and proposes an ISCT credibility assessment workflow based on recently published model assessment frameworks. First, we review various ISCTs described in the literature, carefully selected to showcase the range of methodological options available. These studies cover a wide variety of devices, reasons for conducting ISCTs, patient model generation approaches including subject-specific versus ‘synthetic’ virtual patients, complexity levels of devices and patient models, incorporation of clinician or clinical outcome models, and methods for integrating ISCT results with real-world clinical trials. We next discuss how verification, validation, and uncertainty quantification apply to ISCTs, considering the range of ISCT approaches identified. Based on our analysis, we then present a hierarchical workflow for assessing ISCT credibility, using a general credibility assessment framework recently published by the FDA’s Center for Devices and Radiological Health. Overall, this work aims to promote standardization in ISCTs and contribute to the wider adoption and acceptance of ISCTs as a reliable tool for evaluating medical devices.

Overview

CoChat Summary

TL;DR

This paper reviews eight medical-device in silico clinical trials (ISCTs) and proposes a hierarchical workflow for assessing ISCT credibility based on FDA guidance and ASME V&V40. It argues that ISCT credibility requires sub-model-level verification, validation, and uncertainty quantification, plus population-level validation of the virtual cohort.

Key findings

  • ISCTs are composed of multiple sub-models—baseline patient model, device model, virtual cohort, and optionally clinical outcome, clinician, emulator, and other system models—so credibility assessment should be hierarchical rather than only top-level.
  • The reviewed examples span subject-specific, synthetic, and mixed virtual cohorts: MR-induced heating used 15 subject-specific models from 12 subjects; VICTRE used 2,986 synthetic patients; FD-PASS used 164 virtual patients from 82 anatomies; the artificial pancreas example used 300 synthetic patients; whole-heart CRT models used 24 subject-specific models altered to heart failure; automata-based ICD models used >11,000 virtual patients; and Micra involved 50 million simulations total.
  • Validation can occur at sub-model level (device, patient, coupled device-patient, clinical outcome, clinician) and at cohort level, either individual-level with paired data or population-level without paired data; population-level validation alone risks “right answer for the wrong reason.”
  • The proposed workflow aligns with FDA credibility categories and adds credibility factors/gradations for population-level validation of the virtual cohort, including clinical dataset size, subject range, output comparison rigor, and agreement.

Methods

  • Narrative review of eight selected ISCTs for medical devices, chosen to showcase methodological range rather than to comprehensively survey all ISCTs.
  • Qualitative synthesis of how verification, validation, and uncertainty quantification apply to ISCTs, including sub-model and cohort validation strategies.
  • Proposal of a hierarchical ISCT credibility assessment workflow based on FDA guidance “Assessing the Credibility of Computational Modeling and Simulation in Medical Device Submissions” and ASME V&V40-2018, illustrated with a hypothetical whole-heart/pacemaker-lead ISCT.

Limitations

  • The paper states the workflow may be perceived as overly burdensome, may require case-specific adjustments, and such adjustments could introduce subjectivity.
  • The paper states its scope is first-principles models; data-driven models are not harmonized with first-principles credibility assessment, and ISCT execution is outside scope.
  • The paper states there is no widely accepted community definition of an ISCT, and the literature review is not comprehensive; the proposed workflow also assumes specific features such as a physics-based device model, first-principles patient model, no emulator or explicit clinician model, synthetic cohort, and no Bayesian ISCT/CT integration.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1371/journal.pcbi.1012289

Cites

72 of 72 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Figures

Fig 1 Overview of components of an ISCT (necessary components in white boxes, possible components in light blue boxes). Left side: a virtual cohort (set of virtual patients) can be created given a baseline patient model and a cohort generation method. Coupling with a device model, and potentially other sub-models, results in the virtual cohort for the ISCT, which is run using the ISCT study protocol to obtain the ISCT results. The right-hand box shows four approaches to generating a virtual cohort (not comprehensive). Approach A): Construct subject-specific virtual patients from subject-specific data. B) Construct subject-specific virtual patients and alter to create synthetic patients. C) Generate synthetic patients by sampling from probability distributions derived from subject-specific virtual patients; D) Generate synthetic patients by sampling from prescribed probability distributions. Two parameters, p1 and p2, shown for simplicity. See text for discussion.
Fig 1 Overview of components of an ISCT (necessary components in white boxes, possible components in light blue boxes). Left side: a virtual cohort (set of virtual patients) can be created given a baseline patient model and a cohort generation method. Coupling with a device model, and potentially other sub-models, results in the virtual cohort for the ISCT, which is run using the ISCT study protocol to obtain the ISCT results. The right-hand box shows four approaches to generating a virtual cohort (not comprehensive). Approach A): Construct subject-specific virtual patients from subject-specific data. B) Construct subject-specific virtual patients and alter to create synthetic patients. C) Generate synthetic patients by sampling from probability distributions derived from subject-specific virtual patients; D) Generate synthetic patients by sampling from prescribed probability distributions. Two parameters, p1 and p2, shown for simplicity. See text for discussion.CC0 · PubMed Central
Fig 2 Possibilities for validation of device and patient models (not comprehensive). Equality sign with question mark indicates "Compared as part of validation activity”. See text for discussion.
Fig 2 Possibilities for validation of device and patient models (not comprehensive). Equality sign with question mark indicates "Compared as part of validation activity”. See text for discussion.CC0 · PubMed Central
Fig 3 Possibilities for cohort validation (not comprehensive). Equality sign with question mark indicates "Compared as part of validation activity”. (a) N subject-specific virtual patients each individually validated against data from the corresponding real-world subjects. (b) Distribution of a model output across the virtual cohort validated against corresponding data from the real-world population. (c) Results of an ISCT directly compared to a real-world clinical trial.
Fig 3 Possibilities for cohort validation (not comprehensive). Equality sign with question mark indicates "Compared as part of validation activity”. (a) N subject-specific virtual patients each individually validated against data from the corresponding real-world subjects. (b) Distribution of a model output across the virtual cohort validated against corresponding data from the real-world population. (c) Results of an ISCT directly compared to a real-world clinical trial.CC0 · PubMed Central
Fig 4 Overview of sub-steps in Step 4 of the workflow. Equality sign with question mark indicates "Compared as part of validation activity”.
Fig 4 Overview of sub-steps in Step 4 of the workflow. Equality sign with question mark indicates "Compared as part of validation activity”.CC0 · PubMed Central

Verification

Claims the paper makes

45 Valid · 9 Not valid · 24 Not checked · 105 Own results and interpretations

33 more claims · 6 with their cited sources checked

Create a free account to see all 183 claims and to verify the 2 references still open.

See all claims — free account
  • Probably not a substantial claim. 126
  • It may state two separate claims as one. 49

Create a free account to see these claims.

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

3 of 3

Citations

Cites

Show all 70 references

Cited by

See all 62

Metadata

Authors
Pathmanathan, P., Aycock, K., Badal, A., Bighamian, R., Bodner, J., Craven, B. A., & Niederer, S.
Affiliations
  • United States Food and Drug Administration (US)
  • Medtronic (United States) (US)
  • The Alan Turing Institute (GB)
Corresponding author
Pathmanathan, Pras
Venue
PLOS Computational Biology
Published
2024-08-08
Volume
20
Issue
8
Pages
e1012289
ISSN
  • 1553-734X (ISSN-L)
  • 1553-7358
OpenAlex
W4401418803
Cited by
62
References
70
Citations by year
2026: 30 · 2025: 27 · 2024: 5
FWCI
21.28
Type
journal article
Language
en
Open access
Locations
  • PLoS Computational Biology (publishedVersion, public-domain)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Topics
  • Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
  • Social Sciences › Education › Simulation-Based Education in Healthcare
Keywords
in silico clinical trials, medical devices, credibility assessment, model verification and validation, uncertainty quantification, patient modeling, virtual patients, subject-specific modeling, standardization
MeSH terms
Clinical Trials as Topic, Computer Simulation, Equipment and Supplies, Reproducibility of Results

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Credibility assessment of in silico clinical trials for medical devices. https://app.cochat.ai/papers/10.1371/journal.pcbi.1012289
The paper
Pathmanathan, P., Aycock, K., Badal, A., Bighamian, R., Bodner, J., Craven, B. A., & Niederer, S. (2024). Credibility assessment of in silico clinical trials for medical devices. PLOS Computational Biology. https://doi.org/10.1371/journal.pcbi.1012289

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.