The DOI matches its Crossref record; its reference list is not available.
Protocol for the development of a reporting guideline for causal and counterfactual prediction models in biomedicine
Open accessPublished version of a preprint
Abstract
Introduction: While there are guidelines for reporting on observational studies (eg, Strengthening the Reporting of Observational Studies in Epidemiology, Reporting of Studies Conducted Using Observational Routinely Collected Health Data Statement), estimation of causal effects from both observational data and randomised experiments (eg, A Guideline for Reporting Mediation Analyses of Randomised Trials and Observational Studies, Consolidated Standards of Reporting Trials, PATH) and on prediction modelling (eg, Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis), none is purposely made for deriving and validating models from observational data to predict counterfactuals for individuals on one or more possible interventions, on the basis of given (or inferred) causal structures. This paper describes methods and processes that will be used to develop a Reporting Guideline for Causal and Counterfactual Prediction Models (PRECOG). Methods and analysis: PRECOG will be developed following published guidance from the Enhancing the Quality and Transparency of Health Research (EQUATOR) network and will comprise five stages. Stage 1 will be meetings of a working group every other week with rotating external advisors (active until stage 5). Stage 2 will comprise a systematic review of literature on counterfactual prediction modelling for biomedical sciences (registered in Prospective Register of Systematic Reviews). In stage 3, a computer-based, real-time Delphi survey will be performed to consolidate the PRECOG checklist, involving experts in causal inference, epidemiology, statistics, machine learning, informatics and protocols/standards. Stage 4 will involve the write-up of the PRECOG guideline based on the results from the prior stages. Stage 5 will seek the peer-reviewed publication of the guideline, the scoping/systematic review and dissemination. Ethics and dissemination: The study will follow the principles of the Declaration of Helsinki. The study has been registered in EQUATOR and approved by the University of Florida’s Institutional Review Board (#202200495). Informed consent will be obtained from the working groups and the Delphi survey participants. The dissemination of PRECOG and its products will be done through journal publications, conferences, websites and social media.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
No citation lists for this paper
No citation lists for this paper yet — checked again daily.
References 58 · Cited by 6
Research works printed but not linked: 37
- Van Calster, 2021, Methodology over metrics: current scientific standards are a disservice to patients and society, J Clin Epidemiol, 138, 219, 10.1016/j.jclinepi.2021.05.018 · DOI · Paper page
- 10.1183/13993003.02643-2020 · DOI
- Wilkinson, 2020, Time to reality check the promises of machine learning-powered precision medicine, Lancet Digit Health, 2, e677, 10.1016/S2589-7500(20)30200-4 · DOI · Paper page
- Quinonero-Candela J , Sugiyama M , Schwaighofer A . Dataset shift in machine learning. MIT Press, 2008: 248 p. · DOI · Paper page
- Prosperi, 2020, Causal inference and counterfactual prediction in machine learning for actionable healthcare, Nat Mach Intell, 2, 369, 10.1038/s42256-020-0197-y · DOI · Paper page
- 10.1037/h0037350 · DOI · Paper page
- Pearl J , Glymour M , Jewell NP . Causal inference in statistics: a primer. John Wiley & Sons, 2016: 160 p.
- Nguyen, 2020, Counterfactual clinical prediction models could help to infer individualized treatment effects in randomized controlled trials-An illustration with the International stroke trial, J Clin Epidemiol, 125, 47, 10.1016/j.jclinepi.2020.05.022 · DOI
- 10.1073/pnas.1804597116 · DOI
- 10.1002/sim.4322 · DOI
- 10.1111/j.1541-0420.2012.01763.x · DOI
- Lamont, 2018, Identification of predicted individual treatment effects in randomized clinical trials, Stat Methods Med Res, 27, 142, 10.1177/0962280215623981 · DOI
- Brown K , Merrigan P , Royer J . Estimating average treatment effects with propensity scores estimated with four machine learning procedures: simulation results in high dimensional settings and with time to event outcomes. SSRN Electronic Journal;72.doi:10.2139/ssrn.3272396 · DOI
- Hu, 2021, Estimating heterogeneous survival treatment effects of lung cancer screening approaches: a causal machine learning analysis, Ann Epidemiol, 62, 36, 10.1016/j.annepidem.2021.06.008 · DOI
- Xiong M . Deep learning for causal inference. Artificial Intelligence and Causal Inference 2022:151–208. · DOI
- Ghosh, 2021, Propensity score synthetic augmentation matching using generative adversarial networks (PSSAM-GAN), Comput Methods Programs Biomed Update, 1, 10.1016/j.cmpbup.2021.100020 · DOI
- Ge, 2020, Conditional generative Adversarial networks for individualized treatment effect estimation and treatment selection, Front Genet, 11, 10.3389/fgene.2020.585804 · DOI
- 10.1080/10618600.2017.1356325 · DOI
- 10.1016/j.ijsu.2011.09.004 · DOI · Paper page
- 10.1001/jama.2021.14075 · DOI
- Skrivankova, 2021, Strengthening the reporting of observational studies in epidemiology using Mendelian randomization: the STROBE-MR statement, JAMA, 326, 1614, 10.1001/jama.2021.18236 · DOI · Paper page
- 10.1136/bmj.k3532 · DOI
- 10.1002/pds.1908 · DOI
- Murray EJ , Swanson SA , Hernán MA . Guidelines for estimating causal effects in pragmatic randomized trials. arXiv preprint arXiv:1911.06030 2019. · arXiv
- Baker, 2020, The predictive approaches to treatment effect heterogeneity (path) statement, Ann Intern Med, 172, 775, 10.7326/L20-0426 · DOI
- Moher, 2010, Guidance for developers of health research reporting guidelines, PLoS Med, 7, 10.1371/journal.pmed.1000217 · DOI · Paper page
- Hennink, 2022, Sample sizes for saturation in qualitative research: a systematic review of empirical tests, Soc Sci Med, 292, 114523, 10.1016/j.socscimed.2021.114523 · DOI · Paper page
- Rich, 2020, Employing molecular Phylodynamic methods to identify and forecast HIV transmission clusters in public health settings: a qualitative study, Viruses, 12, 10.3390/v12090921 · DOI
- Lin, 2021, A scoping review of causal methods enabling predictions under hypothetical interventions, Diagn Progn Res, 5, 3, 10.1186/s41512-021-00092-9 · DOI
- Booth A , Clarke M , Dooley G , et al . The nuts and bolts of Prospero: an international prospective register of systematic reviews. Vol. 1, systematic reviews. 2012. Available: http://dx.doi.org/10.1186/2046-4053-1-2 · DOI · Paper page
- Gordon, 2006, RT Delphi: An efficient, “round-less” almost real time Delphi method, Technol Forecast Soc Change, 73, 321, 10.1016/j.techfore.2005.09.005 · DOI
- Hall, 2018, Recruiting and retaining participants in e-Delphi surveys for core outcome set development: evaluating the COMiT'ID study, PLoS One, 13, 10.1371/journal.pone.0201378 · DOI
- Hill, 2011, Bayesian nonparametric modeling for causal inference, Journal of Computational and Graphical Statistics, 20, 217, 10.1198/jcgs.2010.08162 · DOI
- 10.1016/j.im.2003.11.002 · DOI · Paper page
- Hsu, 2007, The Delphi technique: making sense of consensus, Practical Assessment, Research, and Evaluation, 12, 10
- Naughton B , Roberts L , Dopson S , et al . Medicine authentication technology as a counterfeit medicine-detection tool: a Delphi method study to establish expert opinion on manual medicine authentication technology in secondary care. BMJ Open 2017;7:p. e013838.doi:10.1136/bmjopen-2016-013838 · DOI
- Tastle, 2007, Consensus and dissention: a measure of ordinal dispersion, International Journal of Approximate Reasoning, 45, 531, 10.1016/j.ijar.2006.06.024 · DOI
Metadata
- Authors
- Xu, J., Guo, Y., Wang, F., Xu, H., Lucero, R., Bian, J., & Prosperi, M.
- Affiliations
- University of Florida Health (US)
- Cornell University (US)
- The University of Texas Health Science Center at Houston (US)
- University of California, Los Angeles (US)
- University of Florida (US)
- Corresponding author
- Prosperi, Mattia
- Venue
- BMJ Open
- Publisher
- BMJ
- Published
- 2022-06
- Volume
- 12
- Issue
- 6
- Pages
- e059715
- ISSN
- 2044-6055 (ISSN-L)
- PMC
- PMC9214357
- OpenAlex
- W4283167950
- Cited by
- 6
- References
- 58
- Citations by year
- 2026: 1 · 2025: 2 · 2024: 2 · 2023: 1
- FWCI
- 0.75
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- BMJ Open (publishedVersion, cc-by-nc)
- medRxiv (acceptedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by-nc)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (acceptedVersion)
- eScholarship (California Digital Library) (submittedVersion, cc-by-nc)
- Funders
- National Institute on Aging (5R21AG068717-02, R33AG062884-03)
- National Cancer Institute (1R21CA245858-01A1, 1R21CA253394-01A1, 3R01CA246418-02S1, 3R21CA245858- 01A1S1, 5R01CA246418-02)
- Centers for Disease Control and Prevention (U18DP006512)
- University of Florida Informatics Institute Seed grant
- National Institute of Allergy and Infectious Diseases (R01AI141810, R01AI145552)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Advanced Causal Inference Techniques
- Social Sciences › Sociology and Political Science › Delphi Technique in Research
- Keywords
- reporting guidelines, causal inference, observational data, Delphi method, EQUATOR Network, causal structure
- MeSH terms
- Systematic Reviews as Topic, Humans, Research Design, Causality, Guidelines as Topic, Checklist, Observational Studies as Topic
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
