Identifiability and experimental design in perturbation studies
Open accessPublished version of a preprint
The DOI matches its Crossref record; 44 of the 44 references it cites were found in a registry or an index.
Abstract
Motivation: A common strategy to infer and quantify interactions between components of a biological system is to deduce them from the network’s response to targeted perturbations. Such perturbation experiments are often challenging and costly. Therefore, optimizing the experimental design is essential to achieve a meaningful characterization of biological networks. However, it remains difficult to predict which combination of perturbations allows to infer specific interaction strengths in a given network topology. Yet, such a description of identifiability is necessary to select perturbations that maximize the number of inferable parameters. Results: We show analytically that the identifiability of network parameters can be determined by an intuitive maximum-flow problem. Furthermore, we used the theory of matroids to describe identifiability relationships between sets of parameters in order to build identifiable effective network models. Collectively, these results allowed to device strategies for an optimal design of the perturbation experiments. We benchmarked these strategies on a database of human pathways. Remarkably, full network identifiability was achieved, on average, with less than a third of the perturbations that are needed in a random experimental design. Moreover, we determined perturbation combinations that additionally decreased experimental effort compared to single-target perturbations. In summary, we provide a framework that allows to infer a maximal number of interaction strengths with a minimal number of perturbation experiments. Availability and implementation: IdentiFlow is available at github.com/GrossTor/IdentiFlow. Supplementary information: Supplementary data are available at Bioinformatics online.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures



![Fig. 4. Performance of different experimental design strategies on 267 human KEGG pathways. (A) Identifiability AUC, defined as area under the number of identified nodes versus number of perturbation curve, see Equation (13). (B) For each network and strategy, the average number of perturbations required for full identifiability is shown relative to the average number required for a random strategy. (C) The fraction of required perturbations correlated against the isolation score of a network [Equation (14)], r: Spearman’s rank correlation. (D) The fraction between multi-target perturbations with a specific number of targets and all multi-target perturbations (experiments) in KEGG networks of the annotated size range](https://figures.cochat.ai/paper-figures/bcf9ec0d7b514a1ebf708f2ebf393e6d/identifiability-and-experimental-design-in-perturbation-studies-figure-4.jpg)
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Typically, however, perturbation datasets have been too small to elucidate the complexity of cellular systems; thus, accurately predictive models of regulatory interactions remain difficult to infer12.”
From scPerturb: harmonized single-cell perturbation data (2024) · cites this paper as [12]
In the citing paper
“The optimal lower bounds on UMI and cell counts for a given experiment depend on downstream specific modeling tasks, as discussed in more detail elsewhere12.”
From scPerturb: harmonized single-cell perturbation data (2024) · cites this paper as [12]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Causality: Models, Reasoning, and Inference
Pearl, J. (2009) · doi:10.1017/cbo9780511803161 · Cited by 11,653
Network Flows: Theory, Algorithms, and Applications.
Smith, D. K., Ahuja, R. K., Magnanti, T. L., & Orlin, J. B. (1994) · The Journal of the Operational Research Society · doi:10.2307/2583863 · Cited by 8,034
Exploring Network Structure, Dynamics, and Function Using NetworkX
Hagberg, A. A., Schult, D. A., & Swart, P. J. (2008) · Proceedings of the Python in Science Conference · doi:10.25080/tcwv9851 · Cited by 7,477
Theoretical Improvements in Algorithmic Efficiency for Network Flow Problems
Edmonds, J., & Karp, R. M. (1972) · Journal of the ACM · doi:10.1145/321694.321699 · Cited by 2,528
New approach for understanding genome variations in KEGG
Kanehisa, M., Sato, Y., Furumichi, M., Morishima, K., & Tanabe, M. (2019) · Nucleic Acids Research · doi:10.1093/nar/gky962 · Cited by 1,845
Causal Protein-Signaling Networks Derived from Multiparameter Single-Cell Data
Sachs, K., Perez, O., Pe'er, D., Lauffenburger, D. A., & Nolan, G. P. (2005) · Science · doi:10.1126/science.1105809 · Cited by 1,708
Raue, A., Kreutz, C., Maiwald, T., Bachmann, J., Schilling, M., Klingmüller, U., & Timmer, J. (2009) · Bioinformatics · doi:10.1093/bioinformatics/btp358 · Cited by 1,442
Menger, K. (1927) · Fundamenta Mathematicae · doi:10.4064/fm-10-1-96-115 · Cited by 1,216
A Validated Regulatory Network for Th17 Cell Specification
Ciofani, M., Madar, A., Galan, C., Sellars, M., Mace, K., Pauli, F., Agarwal, A., Huang, W., Parkurst, C. N., Muratet, M., et al. (2012) · Cell · doi:10.1016/j.cell.2012.09.016 · Cited by 1,160
Inferring Genetic Networks and Identifying Compound Mode of Action via Expression Profiling
Gardner, T. S., di Bernardo, D., Lorenz, D., & Collins, J. J. (2003) · Science · doi:10.1126/science.1081900 · Cited by 1,117
Pooled CRISPR screening with single-cell transcriptome readout.
Datlinger, P., Rendeiro, A. F., Schmidl, C., Krausgruber, T., Traxler, P., Klughammer, J., Schuster, L. C., Kuchler, A., Alpar, D., & Bock, C. (2017) · Nature Methods · doi:10.1038/nmeth.4177 · Cited by 1,096
Algorithm for solution of a problem of maximum flow in networks with power estimation
Dinic, E. A. (1970) · Medical Entomology and Zoology · Cited by 1,020
Dissecting Immune Circuits by Linking CRISPR-Pooled Screens with Single-Cell RNA-Seq
Jaitin, D. A., Weiner, A., Yofe, I., Lara-Astiaso, D., Keren-Shaul, H., David, E., Salame, T. M., Tanay, A., van Oudenaarden, A., & Amit, I. (2016) · Cell · doi:10.1016/j.cell.2016.11.039 · Cited by 777
Revealing strengths and weaknesses of methods for gene network inference
Marbach, D., Prill, R. J., Schaffter, T., Mattiussi, C., Floreano, D., & Stolovitzky, G. (2010) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0913357107 · Cited by 752
On the Abstract Properties of Linear Dependence
Whitney, H. (1935) · American Journal of Mathematics · doi:10.2307/2371182 · Cited by 713
On the Abstract Properties of Linear Dependence1
Whitney, H. (2009) · Classic Papers in Combinatorics · doi:10.1007/978-0-8176-4842-8_5 · Cited by 653
Advantages and limitations of current network inference methods
De Smet, R., & Marchal, K. (2010) · Nature Reviews Microbiology · doi:10.1038/nrmicro2419 · Cited by 515
Bonneau, R., Reiss, D. J., Shannon, P., Facciotti, M., Hood, L., Baliga, N. S., & Thorsson, V. (2006) · Genome Biology · doi:10.1186/gb-2006-7-5-r36 · Cited by 510
Reverse engineering gene networks: integrating genetic perturbations with dynamical modeling
Tegnér, J., Yeung, M. K. S., Hasty, J., & Collins, J. J. (2003) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0933416100 · Cited by 433
Revealing Network Connectivity from Response Dynamics
Timme, M. (2007) · Physical Review Letters · doi:10.1103/physrevlett.98.224101 · Cited by 333
Even, S. (2011) · Cambridge University Press eBooks · doi:10.1017/cbo9781139015165 · Cited by 218
Strong negative feedback from Erk to Raf confers robustness to MAPK signalling
Fritsche‐Guenther, R., Witzel, F., Sieber, A., Herr, R., Schmidt, N., Braun, S., Brummer, T., Sers, C., & Blüthgen, N. (2011) · Molecular Systems Biology · doi:10.1038/msb.2011.27 · Cited by 192
Self-citationAn experimentally supported model of the Bacillus subtilis global transcriptional regulatory network
Arrieta‐Ortiz, M. L., Hafemeister, C., Bate, A. R., Chu, T., Greenfield, A., Shuster, B., Barry, S. N., Gallitto, M., Liu, B., Kacmarczyk, T., et al. (2015) · Molecular Systems Biology · doi:10.15252/msb.20156236 · Cited by 186
Identifiability of Model Parameter
Godfrey, K. R., & DiStefano, J. J. (1985) · IFAC Proceedings Volumes · doi:10.1016/s1474-6670(17)60544-5 · Cited by 170
Data-based identifiability analysis of non-linear dynamical models
Hengl, S., Kreutz, C., Timmer, J., & Maiwald, T. (2007) · Bioinformatics · doi:10.1093/bioinformatics/btm382 · Cited by 166
Network quantification of EGFR signaling unveils potential for targeted combination therapy
Klinger, B., Sieber, A., Fritsche‐Guenther, R., Witzel, F., Berry, L., Schumacher, D., Yan, Y., Durek, P., Merchant, M., Schäfer, R., et al. (2013) · Molecular Systems Biology · doi:10.1038/msb.2013.29 · Cited by 155
Self-citationAddressing parameter identifiability by model-based experimentation
Raue, A., Kreutz, C., Maiwald, T., Klingmüller, U., & Timmer, J. (2011) · IET Systems Biology · doi:10.1049/iet-syb.2010.0061 · Cited by 145
Greenfield, A., Hafemeister, C., & Bonneau, R. (2013) · Bioinformatics · doi:10.1093/bioinformatics/btt099 · Cited by 141
Modular Response Analysis of Cellular Regulatory Networks
Bruggeman, F. J., Westerhoff, H. V., Hoek, J. B., & Kholodenko, B. Ν. (2002) · Journal of Theoretical Biology · doi:10.1006/jtbi.2002.3096 · Cited by 140
Methods for causal inference from gene perturbation experiments and validation
Meinshausen, N., Hauser, A., Mooij, J. M., Peters, J., Versteeg, P., & Bühlmann, P. (2016) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1510493113 · Cited by 140
CRISPR/Cas9-Enabled Multiplex Genome Editing and Its Application
Minkenberg, B., Wheatley, M. S., & Yang, Y. (2017) · Progress in molecular biology and translational science · doi:10.1016/bs.pmbts.2017.05.003 · Cited by 120
Modular Response Analysis of Cellular Regulatory Networks
Bruggeman, F. J. (2002) · Journal of Theoretical Biology · doi:10.1016/s0022-5193(02)93096-1 · Cited by 119
Brandt, R., Sell, T., Lüthen, M., Uhlitz, F., Klinger, B., Riemer, P., Giesecke-Thiel, C., Schulze, S., El-Shimy, I. A., Kunkel, D., et al. (2019) · Nature Communications · doi:10.1038/s41467-019-10954-y · Cited by 91
Self-citationA network biology approach to aging in yeast
Lorenz, D. R., Cantor, C. R., & Collins, J. J. (2009) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0812551106 · Cited by 78
Untangling the signalling wires
Kholodenko, B. Ν. (2007) · Nature Cell Biology · doi:10.1038/ncb0307-247 · Cited by 73
Discriminating direct and indirect connectivities in biological networks
Kang, T., Moore, R. L., Li, Y., Sontag, E. D., & Bleris, L. G. (2015) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1507168112 · Cited by 42
Santra, T., Kölch, W., & Kholodenko, B. Ν. (2013) · BMC Systems Biology · doi:10.1186/1752-0509-7-57 · Cited by 34
Modelling signalling networks from perturbation data
Dorel, M., Klinger, B., Groß, T., Sieber, A., Prahallad, A., Bosdriesz, E., Wessels, L. F. A., & Blüthgen, N. (2018) · Bioinformatics · doi:10.1093/bioinformatics/bty473 · Cited by 30
Self-citationParadoxical Results in Perturbation-Based Signaling Network Reconstruction
Prabakaran, S., Gunawardena, J., & Sontag, E. D. (2014) · Biophysical Journal · doi:10.1016/j.bpj.2014.04.031 · Cited by 28
Reconstructing static and dynamic models of signaling pathways using Modular Response Analysis
Santra, T., Rukhlenko, O. S., Zhernovkov, V., & Kholodenko, B. Ν. (2018) · Current Opinion in Systems Biology · doi:10.1016/j.coisb.2018.02.003 · Cited by 28
Reverse engineering gene regulatory networks by modular response analysis – a benchmark
Klinger, B., & Blüthgen, N. (2018) · Essays in Biochemistry · doi:10.1042/ebc20180012 · Cited by 19
Self-citationAlgorithms for Enumerating Circuits in Matroids
Boros, E., Elbassioni, K., Gurvich, V., et al. (2003) · Lecture notes in computer science · doi:10.1007/978-3-540-24587-2_50 · Cited by 11
Comparative Network Reconstruction using mixed integer programming
Bosdriesz, E., Prahallad, A., Klinger, B., Sieber, A., Bosma, A. R., Bernards, R., Blüthgen, N., & Wessels, L. F. A. (2018) · Bioinformatics · doi:10.1093/bioinformatics/bty616 · Cited by 11
Self-citationRobust network inference using response logic
Gross, T., Wongchenko, M. J., Yan, Y., & Blüthgen, N. (2019) · Bioinformatics · doi:10.1093/bioinformatics/btz326 · Cited by 7
Self-citation
Research works printed but not linked: 1
- Minkenberg, 2017, Progress in Molecular Biology and Translational Science. Gene Editing in Plants, 111
Cited by
On structural and practical identifiability
Wieland, F.-G., Hauber, A. L., Rosenblatt, M., Tönsing, C., & Timmer, J. (2021) · Current Opinion in Systems Biology · doi:10.1016/j.coisb.2021.03.005 · Cited by 325
scPerturb: harmonized single-cell perturbation data
Peidli, S., Green, T. D., Shen, C., Gross, T., Min, J., Garda, S., Yuan, B., Schumacher, L. J., Taylor-King, J. P., Marks, D. S., et al. (2024) · Nature Methods · doi:10.1038/s41592-023-02144-y · Cited by 194
Self-citationRood, J. E., Hupalowska, A., & Regev, A. (2024) · Cell · doi:10.1016/j.cell.2024.07.035 · Cited by 139
Metadata
- Authors
- Gross, T., & Blüthgen, N.
- Affiliations
- Humboldt-Universität zu Berlin (DE)
- Corresponding author
- Blüthgen, Nils
- ORCID
- Blüthgen, Nils
- Venue
- Bioinformatics
- Publisher
- Oxford University Press (OUP)
- Published
- 2020-07-01
- Volume
- 36
- Issue
- Supplement_1
- Pages
- i482-i489
- ISSN
- 1367-4803 (ISSN-L)
- 1367-4811
- PMC
- PMC7355299
- OpenAlex
- W3042055820
- Cited by
- 11
- References
- 44
- Citations by year
- 2026: 1 · 2024: 3 · 2023: 1 · 2022: 1 · 2021: 4 · 2020: 1
- FWCI
- 0.57
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Bioinformatics (publishedVersion, cc-by-nc)
- bioRxiv (Cold Spring Harbor Laboratory) (acceptedVersion, cc-by-nc-nd)
- PubMed Central (publishedVersion, cc-by-nc)
- PubMed (publishedVersion)
- edoc Publication server (Humboldt University of Berlin) (submittedVersion, cc-by-nc)
- edoc Publication server (Humboldt University of Berlin)
- Funders
- Deutsche Forschungsgemeinschaft (RTG2424)
- Topics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Gene Regulatory Network Analysis
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Bioinformatics and Genomic Networks
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Microbial Metabolic Engineering and Bioproduction
- Keywords
- identifiability, experimental design, perturbation experiments, biological networks, maximum flow, matroid theory, network parameter estimation
- MeSH terms
- Humans, Models, Biological, Research Design
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.