Including non‐randomized studies on intervention effects
The DOI matches its Crossref record; 14 of the 14 references it cites were found in a registry or an index.
Abstract
This chapter aims to support review authors who are considering including non-randomized studies of interventions (NRSI) in a Cochrane Review. NRSI are defined here as any quantitative study estimating the effectiveness of an intervention (harm or benefit) that does not use randomization to allocate units (individuals or clusters of individuals) to intervention groups. The chapter has been prepared by the Cochrane Non-Randomized Studies of Interventions Methods Group. It aims to describe the particular challenges that arise if NRSI are included in a Cochrane Review. The chapter recommends that eligibility criteria, data collection and assessment of included studies place an emphasis on specific features of study design rather than ‘labels’ for study designs. Review authors should consider how potential confounders, and how the likelihood of increased heterogeneity resulting from residual confounding and from other biases that vary across studies, are addressed in meta-analyses of non-randomized studies.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions
Sterne, J. A., Hernán, M. A., Reeves, B. C., Savović, J., Berkman, N. D., Viswanathan, M., Henry, D., Altman, D. G., Ansari, M. T., Boutron, I., et al. (2016) · BMJ · doi:10.1136/bmj.i4919 · Cited by 20,329
Self-citationThe Environment and Disease: Association or Causation?
Hill, A. B. (1965) · Proceedings of the Royal Society of Medicine · doi:10.1177/003591576505800503 · Cited by 6,974
Moher, D., Schulz, K. F., & Altman, D. G. (2001) · The Lancet · doi:10.1016/s0140-6736(00)04337-3 · Cited by 5,373
Wohlin, C. (2014) · Proceedings of the 18th International Conference on Evaluation and Assessment in Software Engineering · doi:10.1145/2601248.2601268 · Cited by 4,234
Evaluating non-randomised intervention studies
Deeks, J., Dinnes, J., D'Amico, R., Sowden, A., Sakarovitch, C., Song, F., Petticrew, M., & Altman, D. (2003) · Health Technology Assessment · doi:10.3310/hta7270 · Cited by 2,907
Self-citationChan, A.-W., Hróbjartsson, A., Haahr, M. T., Gøtzsche, P. C., & Altman, D. G. (2004) · JAMA · doi:10.1001/jama.291.20.2457 · Cited by 1,845
Schünemann, H. J., Cuello, C., Akl, E. A., Mustafa, R. A., Meerpohl, J. J., Thayer, K., Morgan, R. L., Gartlehner, G., Kunz, R., Katikireddi, S. V., et al. (2019) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2018.01.012 · Cited by 911
Self-citationLarge-scale randomized evidence: Large, simple trials and overviews of trials
Peto, R., Collins, R., & Gray, R. (1995) · Journal of Clinical Epidemiology · doi:10.1016/0895-4356(94)00150-o · Cited by 365
MacLehose, Reeves, Harvey, Sheldon, Russell, & Black (2000) · Health Technology Assessment · doi:10.3310/hta4340 · Cited by 311
Schünemann, H. J., Tugwell, P., Reeves, B. C., Akl, E. A., Santesso, N., Spencer, F. A., Shea, B., Wells, G., & Helfand, M. (2013) · Research Synthesis Methods · doi:10.1002/jrsm.1078 · Cited by 248
Self-citation
Cited by
Barteit, S., Lanfermann, L., Bärnighausen, T., Neuhann, F., & Beiersmann, C. (2021) · JMIR Serious Games · doi:10.2196/29080 · Cited by 547
Piechotta, V., Chai, K. L., Valk, S. J., Doree, C., Monsef, I., Wood, E. M., Lamikanra, A., Kimber, C., McQuilten, Z., So-Osman, C., et al. (2020) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd013600.pub2 · Cited by 367
Insfran-Rivarola, A., Tlapa, D., Limon-Romero, J., Baez-Lopez, Y., Miranda-Ackerman, M., Arredondo-Soto, K., & Ontiveros, S. (2020) · Foods · doi:10.3390/foods9091169 · Cited by 160
Castellana, M., Parisi, C., Di Molfetta, S., Di Gioia, L., Natalicchio, A., Perrini, S., Cignarelli, A., Laviola, L., & Giorgino, F. (2020) · BMJ Open Diabetes Research & Care · doi:10.1136/bmjdrc-2019-001092 · Cited by 84
“42 Also, non-randomized studies should not be combined with randomized studies in a meta-analysis given the very different design features.”
Kristjansson, E., Dignam, M., Rizvi, A., Osman, M., Magwood, O., Olarte, D., Cohen, J. F., Krasevec, J., Grover, T., Labelle, P. R., et al. (2025) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd014794.pub2 · Cited by 4
Self-citation
Metadata
- Authors
- Reeves, B. C., Deeks, J. J., Higgins, J. P., Shea, B., Tugwell, P., Wells, G. A., & on behalf of the Cochrane Non‐Randomized Studies of Interventions Methods Group
- Venue
- Cochrane Handbook for Systematic Reviews of Interventions
- Publisher
- Wiley
- Published
- 2019-09-20
- Pages
- 595-620
- OpenAlex
- W2974521163
- Cited by
- 153
- References
- 14
- Type
- other
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Including non‐randomized studies on intervention effects. https://app.cochat.ai/papers/10.1002/9781119536604.ch24
- The paper
- Reeves, B. C., Deeks, J. J., Higgins, J. P., Shea, B., Tugwell, P., Wells, G. A., & on behalf of the Cochrane Non‐Randomized Studies of Interventions Methods Group. (2019). Including non‐randomized studies on intervention effects. Cochrane Handbook for Systematic Reviews of Interventions. https://doi.org/10.1002/9781119536604.ch24
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.