CoChat

Relating simulation studies by provenance—Developing a family of Wnt signaling models

Open access
Budde, Kai, Smith, Jacob, Wilsdorf, Pia, Haack, Fiete, Uhrmacher, Adelinde M. · 2021 · PLOS Computational Biology

Published version of a preprint

The DOI matches its Crossref record; 114 of the 115 references it cites were found in a registry or an index; 54 of the 143 sources its claims cite were checked for support.

Abstract

For many biological systems, a variety of simulation models exist. A new simulation model is rarely developed from scratch, but rather revises and extends an existing one. A key challenge, however, is to decide which model might be an appropriate starting point for a particular problem and why. To answer this question, we need to identify entities and activities that contributed to the development of a simulation model. Therefore, we exploit the provenance data model, PROV-DM, of the World Wide Web Consortium and, building on previous work, continue developing a PROV ontology for simulation studies. Based on a case study of 19 Wnt/β-catenin signaling models, we identify crucial entities and activities as well as useful metadata to both capture the provenance information from individual simulation studies and relate these forming a family of models. The approach is implemented in WebProv , a web application for inserting and querying provenance information. Our specialization of PROV-DM contains the entities Research Question, Assumption, Requirement, Qualitative Model, Simulation Model, Simulation Experiment, Simulation Data, and Wet-lab Data as well as activities referring to building, calibrating, validating, and analyzing a simulation model. We show that most Wnt simulation models are connected to other Wnt models by using (parts of) these models. However, the overlap, especially regarding the Wet-lab Data used for calibration or validation of the models is small. Making these aspects of developing a model explicit and queryable is an important step for assessing and reusing simulation models more effectively. Exposing this information helps to integrate a new simulation model within a family of existing ones and may lead to the development of more robust and valid simulation models. We hope that our approach becomes part of a standardization effort and that modelers adopt the benefits of provenance when considering or creating simulation models.

Overview

CoChat Summary

TL;DR

The paper develops a PROV-DM specialization for capturing provenance of simulation studies and uses it to relate 19 Wnt/β-catenin signaling models into a family. It shows most Wnt models are connected through reuse of models or model parts, while overlap in wet-lab data used for calibration or validation is small.

Key findings

  • The PROV-DM specialization defines entities Research Question, Assumption, Requirement, Qualitative Model, Simulation Model, Simulation Experiment, Simulation Data, and Wet-lab Data, plus activities Building, Calibrating, Validating, and Analyzing Simulation Model, connected via `wasGeneratedBy` and `used`.
  • In the case study of 19 Wnt/β-catenin simulation studies, all but two models are indirectly connected to Lee et al. (2003); the exceptions are Sick et al. (2006) and Rodríguez-González et al. (2007).
  • Wet-lab data overlap is small: wet-lab data from just three studies ([105–107]) were reused by simulation studies, and only Lee et al. (2003), Kogan et al. (2012), and Haack et al. (2015) both calibrated and validated models based on wet-lab data.
  • Of 106 collected assumptions, the three most used SBO categories were kinetic constants (13), transport (9), and equivalence (8); of 145 simulation experiments, most were parameter scans, followed by time course analyses and perturbations.

Methods

  • Design: refine a PROV-DM ontology for cellular biochemical simulation studies, implemented in WebProv, a web application with a Vue/D3 frontend and Neo4j graph backend for storing, displaying, and querying provenance.
  • Sample: from 31 found Wnt simulation studies, 19 were selected—all Wnt studies stored in BioModels (6), all published by the authors’ group (4), and nine selected randomly—plus referenced publications used by those studies.
  • Analysis: provenance entities, activities, and relations were manually collected from publications and supporting materials; assumptions were categorized using the Systems Biology Ontology, simulation experiments were categorized by type, and provenance graphs were queried to identify relationships among models.

Limitations

  • The paper states that manual provenance collection is demanding and requires interpretation because natural-language descriptions are ambiguous and nonlinear; this likely hampers text mining or machine learning replacement, and provenance should ideally be collected during the simulation study.
  • The paper states that identifying assumptions involves uncertainty: authors may not state all assumptions, and the authors of this paper may have missed or over-interpreted some; two of 19 studies did not explicitly state assumptions.
  • The paper states that Requirements were often not explicit and were identified only in four publications; the approach is coarse-grained, omits PROV-DM agent types and relations such as `WasInformedBy` and `WasDerivedFrom`, and some simulation or wet-lab data were not shown or lacked references.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1371/journal.pcbi.1009227

Cites

114 of 114 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Figures

Fig 1 Combined overview of all qualitative Wnt (sub-)models found within the 19 Wnt simulation studies. Depicted are the components (species, compartments, and reactions) of the canonical Wnt signaling pathway (a) and its crosstalk with other signaling pathways (b) found within the 19 Wnt simulation studies. Note that the overview is a simplified and condensed representation. Interactions are simplified and some components of the submodels that do not directly affect the Wnt signaling pathway are omitted. Activated/phosphorylated proteins are indicated by (*). Inactive/unphosphorylated states of proteins have been omitted when possible. Submodels involving membrane-mediated processes, such as receptor/ligand interactions, destruction complex recruitment and endocytosis [31, 45, 51, 56], or cadherin-mediated cell adhesion [34, 39, 49] are incorporated in (a). Submodels involving crosstalk with ERK/FGF/PI3K/Akt [32, 36, 53], Notch [33, 36, 48], and ROS/Dvl-mediated pathways [51, 57] are shown in the lower panels of (b), respectively.
Fig 1 Combined overview of all qualitative Wnt (sub-)models found within the 19 Wnt simulation studies. Depicted are the components (species, compartments, and reactions) of the canonical Wnt signaling pathway (a) and its crosstalk with other signaling pathways (b) found within the 19 Wnt simulation studies. Note that the overview is a simplified and condensed representation. Interactions are simplified and some components of the submodels that do not directly affect the Wnt signaling pathway are omitted. Activated/phosphorylated proteins are indicated by (*). Inactive/unphosphorylated states of proteins have been omitted when possible. Submodels involving membrane-mediated processes, such as receptor/ligand interactions, destruction complex recruitment and endocytosis [31, 45, 51, 56], or cadherin-mediated cell adhesion [34, 39, 49] are incorporated in (a). Submodels involving crosstalk with ERK/FGF/PI3K/Akt [32, 36, 53], Notch [33, 36, 48], and ROS/Dvl-mediated pathways [51, 57] are shown in the lower panels of (b), respectively.CC BY · PubMed Central
Fig 2 UML class diagram of provenance entities in WebProv. We have identified the following entities to be useful for providing provenance information from simulation studies in the field of systems biology: Research Question, Assumption, Requirement, Qualitative Model, Simulation Model, Simulation Experiment, Simulation Data, and Wet-lab Data. The requested information for each entity type is kept minimal for demonstration purposes and can easily be extended. The Study (Reference) contains information of the publication, for instance, “Lee et al. (2003)” and determines which study an entity belongs to. The Description contains a brief explanation of a particular entity and may be a cited text from the publication. Furthermore, entity references should ideally consist of a digital object identifier (DOI) to make the artifact associated with the particular entity unambiguously accessible. Additional information can always be entered in the “Further Information” part of WebProv.
Fig 2 UML class diagram of provenance entities in WebProv. We have identified the following entities to be useful for providing provenance information from simulation studies in the field of systems biology: Research Question, Assumption, Requirement, Qualitative Model, Simulation Model, Simulation Experiment, Simulation Data, and Wet-lab Data. The requested information for each entity type is kept minimal for demonstration purposes and can easily be extended. The Study (Reference) contains information of the publication, for instance, “Lee et al. (2003)” and determines which study an entity belongs to. The Description contains a brief explanation of a particular entity and may be a cited text from the publication. Furthermore, entity references should ideally consist of a digital object identifier (DOI) to make the artifact associated with the particular entity unambiguously accessible. Additional information can always be entered in the “Further Information” part of WebProv.CC BY · PubMed Central
Fig 3 Provenance graph of the study by Lee et al. (2003) [28]. Besides the entities and activities that make up the provenance information from the study (see legend), additional entities from three other studies [64–66], which were used by Lee et al. (2003), are shown. The colors of the ellipses show different entity types, the borders of the rectangles visualize different activity types. The gray areas separate the individual studies. The graph displays, for example, that the Building Simulation Model activity BSM1 used, among others, the entity WD1 of type Wet-lab Data from Lee et al. 2001 [64]. This activity then generated the Simulation Model SM1.
Fig 3 Provenance graph of the study by Lee et al. (2003) [28]. Besides the entities and activities that make up the provenance information from the study (see legend), additional entities from three other studies [64–66], which were used by Lee et al. (2003), are shown. The colors of the ellipses show different entity types, the borders of the rectangles visualize different activity types. The gray areas separate the individual studies. The graph displays, for example, that the Building Simulation Model activity BSM1 used, among others, the entity WD1 of type Wet-lab Data from Lee et al. 2001 [64]. This activity then generated the Simulation Model SM1.CC BY · PubMed Central
Fig 4 Screenshot of WebProv. This screenshot shows the provenance graph of the study by Lee et al. (2003) [28] with additional entities from three other studies [64–66], which are automatically colored differently. The node SD2 has been clicked on, which opens a box on the right with the stored and editable metadata.
Fig 4 Screenshot of WebProv. This screenshot shows the provenance graph of the study by Lee et al. (2003) [28] with additional entities from three other studies [64–66], which are automatically colored differently. The node SD2 has been clicked on, which opens a box on the right with the stored and editable metadata.CC BY · PubMed Central
Fig 5 Categories of the simulation experiments conducted within the analyzed 19 Wnt signaling studies. All of these simulation experiments have been categorized. Most simulation experiments were parameter scans.
Fig 5 Categories of the simulation experiments conducted within the analyzed 19 Wnt signaling studies. All of these simulation experiments have been categorized. Most simulation experiments were parameter scans.CC BY · PubMed Central
Fig 6 Provenance graph of all Wnt/β-catenin simulation studies considered here (black outlines) as well as additionally used studies (gray outlines). The colors indicate the cell lines used in wet-lab experiments of that study (see legend below graph). Gray boxes represent pure Wnt/β-catenin simulation studies without acquiring wet-lab data. White boxes display publications used by some of the Wnt/β-catenin simulation studies that are either text books or simulation studies without published wet-lab data. The figure was created using the R package DiagrammeR [103]. The references to the additionally used studies are found in S1 Appendix.
Fig 6 Provenance graph of all Wnt/β-catenin simulation studies considered here (black outlines) as well as additionally used studies (gray outlines). The colors indicate the cell lines used in wet-lab experiments of that study (see legend below graph). Gray boxes represent pure Wnt/β-catenin simulation studies without acquiring wet-lab data. White boxes display publications used by some of the Wnt/β-catenin simulation studies that are either text books or simulation studies without published wet-lab data. The figure was created using the R package DiagrammeR [103]. The references to the additionally used studies are found in S1 Appendix.CC BY · PubMed Central

Verification

Claims the paper makes

29 Valid · 6 Not valid · 27 Not checked · 76 Own results and interpretations

57 more claims · 12 with their cited sources checked

Create a free account to see all 138 claims and to verify the 19 references still open.

See all claims — free account
  • It may state two separate claims as one. 32
  • Probably not a substantial claim. 59

Create a free account to see these claims.

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

2 of 2

Citations

Cites

Show all 114 references

Research works printed but not linked: 1

  1. D Peng, 2016, Reusing simulation experiment specifications to support developing models by successive extension, Simulation Modelling Practice and Theory, 68, 33, 10.1016/j.simpat.2016.07.006 · DOI

Cited by

See all 23

Metadata

Authors
Budde, K., Smith, J., Wilsdorf, P., Haack, F., & Uhrmacher, A. M.
Affiliations
  • University of Rostock (DE)
  • University of New Brunswick (CA)
Corresponding author
Budde, Kai
Venue
PLOS Computational Biology
Published
2021-08-05
Volume
17
Issue
8
Pages
e1009227
ISSN
  • 1553-734X (ISSN-L)
  • 1553-7358
OpenAlex
W3191637716
Cited by
23
References
115
Citations by year
2026: 1 · 2025: 6 · 2024: 5 · 2023: 2 · 2022: 4 · 2021: 5
FWCI
3.42
Type
journal article
Language
en
Open access
Locations
  • PLoS Computational Biology (publishedVersion, cc-by)
  • bioRxiv (Cold Spring Harbor Laboratory) (acceptedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Funders
  • Deutsche Forschungsgemeinschaft (SFB 1270/1—299150580, 320435134)
  • Deutscher Akademischer Austauschdienst (57467143)
Topics
  • Social Sciences › Decision Sciences › Information Systems and Management › Scientific Computing and Data Management
  • Social Sciences › Arts and Humanities › History and Philosophy of Science › Philosophy of Biology and Scientific Explanation
  • Physical Sciences › Computer Science › Information Systems › Research Data Management Practices
Keywords
Wnt/β-catenin signaling, model reuse, model calibration, model validation, model families, model standardization
MeSH terms
Animals, Biochemical Phenomena, Computer Graphics, Computer Simulation, Humans, Models, Biological, Software, Computational Biology, Systems Biology, Wnt Signaling Pathway

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Relating simulation studies by provenance—Developing a family of Wnt signaling models. https://app.cochat.ai/papers/10.1371/journal.pcbi.1009227
The paper
Budde, K., Smith, J., Wilsdorf, P., Haack, F., & Uhrmacher, A. M. (2021). Relating simulation studies by provenance—Developing a family of Wnt signaling models. PLOS Computational Biology. https://doi.org/10.1371/journal.pcbi.1009227

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.