COSMOS-E: Guidance on conducting systematic reviews and meta-analyses of observational studies of etiology
Open accessThe DOI matches its Crossref record; 105 of the 105 references it cites were found in a registry or an index; 10 of the 104 sources its claims cite were checked for support.
Abstract
BACKGROUND: To our knowledge, no publication providing overarching guidance on the conduct of systematic reviews of observational studies of etiology exists. METHODS AND FINDINGS: Conducting Systematic Reviews and Meta-Analyses of Observational Studies of Etiology (COSMOS-E) provides guidance on all steps in systematic reviews of observational studies of etiology, from shaping the research question, defining exposure and outcomes, to assessing the risk of bias and statistical analysis. The writing group included researchers experienced in meta-analyses and observational studies of etiology. Standard peer-review was performed. While the structure of systematic reviews of observational studies on etiology may be similar to that for systematic reviews of randomised controlled trials, there are specific tasks within each component that differ. Examples include assessment for confounding, selection bias, and information bias. In systematic reviews of observational studies of etiology, combining studies in meta-analysis may lead to more precise estimates, but such greater precision does not automatically remedy potential bias. Thorough exploration of sources of heterogeneity is key when assessing the validity of estimates and causality. CONCLUSION: As many reviews of observational studies on etiology are being performed, this document may provide researchers with guidance on how to conduct and analyse such reviews.
Overview
CoChat Summary
TL;DR
COSMOS-E is an expert guidance document for conducting systematic reviews and meta-analyses of observational studies of etiology, covering all steps from question formulation to interpretation. It stresses that meta-analysis may improve precision but does not automatically remedy confounding, selection bias, or information bias, so heterogeneity and risk of bias must be explored.
Key findings
- No overarching guidance on systematic reviews of observational studies of etiology existed to the authors’ knowledge; COSMOS-E addresses all review steps, including PECO definition, protocol, search, data extraction, bias assessment, meta-analysis, and interpretation.
- In 2014, of about 8,000 systematic reviews published, 36% were on etiology, prognosis, or diagnosis; reviews of observational studies typically include 10 to 20 studies.
- Bias assessment should distinguish confounding, selection bias, information bias, and study sensitivity; more than 80 tools exist for assessing risk of bias in nonrandomised studies, and 30%–50% of studies indexed as “case-control studies” in specialty journals are not actually case-control studies.
- Combining studies in meta-analysis may lead to more precise estimates, but “greater precision does not automatically remedy potential bias”; thorough exploration of heterogeneity is key when assessing validity and causality.
Methods
- Guidance was developed by a writing group of researchers experienced in meta-analyses and observational studies of etiology; no external advice was sought, and standard peer-review was performed.
- The document is not formally a guideline and does not prescribe how researchers must perform or report reviews; it covers basic principles and advanced topics but excludes systematic reviews of nonrandomised studies of interventions, or of diagnostic, prognostic, or genetic studies.
- No empirical data collection or meta-analysis was performed; recommendations are based on theoretical considerations, empirical work, and experience with assessing risk of bias in RCTs and other studies.
Limitations
- The paper states that COSMOS-E is not formally meant to be a guideline, does not formally prescribe conduct or reporting, and will not settle ongoing debates and controversies around reviews of observational studies of etiology.
- The paper states that no external advice was sought and that standard peer-review was performed.
- Evident from the text: the recommendations are expert guidance rather than a systematic review or empirical validation of methods, and the authors note that the “roadmap” for observational reviews is less standardised and some design issues are not settled.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures
![Fig 1 Flow chart of study selection. From [15].](https://figures.cochat.ai/paper-figures/b0d6cff55ef043e1b23baa2d5394d204/cosmos-e-guidance-on-conducting-systematic-reviews-and-meta-analyses-of-figure-1.jpg)

![Fig 3 Illustration of the ecological fallacy. Hypothetical example of aggregate and individual level CD4 cell count data at the start of ART. Adapted from [83]. ART, antiretroviral therapy.](https://figures.cochat.ai/paper-figures/fef53086551b4ddd9ed09136439a4a89/cosmos-e-guidance-on-conducting-systematic-reviews-and-meta-analyses-of-figure-3.jpg)
Verification
Claims the paper makes
9 Valid · 8 Not valid · 78 Not checked · 73 Own results and interpretations
Every source the paper cites for this claim was checked and supports it.
In the paper
“A related issue arises when the selection of the study population or statistical model is chosen based on the P value (‘P hacking’) [64].”
§ COSMOS-E: Guidance on conducting systematic reviews and meta-analyses of observational studies of etiology > Assessing quality and bias > Reporting biases, P hacking, and analytic choices
What the sources say
- Supportedcites [64] ML Head, 2015, The extent and consequences of p-hacking in science, PLoS Biol source
The paper’s discussion of p-hacking (section and page not identified in our copy) lists post-analysis decisions to drop outliers, change treatment groups, and include or exclude covariates. It also describes stopping data exploration when an analysis yields a significant p-value. These examples support the claim that choosing a study population or statistical model based on p-values is p-hacking. (LLM, conf=0.92)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“All of these models, including more advanced techniques such as propensity scores [46] or inverse probability weighting, rely on the assumption of no unmeasured confounding for the validity of the effect estimate.”
§ COSMOS-E: Guidance on conducting systematic reviews and meta-analyses of observational studies of etiology > Assessing quality and bias > Risk of bias in individual studies: Confounding, selection bias, and information bias
What the sources say
- Supportedcites [46] PC Austin, 2011, An Introduction to Propensity Score Methods for Reducing the Effects of Confounding in Observational Studies, Multivariate Behav Res source
The passage in 'THE PROPENSITY SCORE AND PROPENSITY SCORE METHODS' states that the first condition of strong ignorability is the 'no unmeasured confounders' assumption and that 'this is the crucial assumption that underlies propensity score analyses', conditioning on the score yielding unbiased average treatment effects only when it holds. The same section lists inverse probability of treatment weighting as one of the four propensity score methods (also in its IPTW subsection), so IPTW falls within what the assumption underlies; the passage adds that the assumption also underlies regression-based approaches, matching the citing sentence's broader framing about statistical models for confounding adjustment. (LLM, conf=0.90)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption not shown here.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“The studies included in a systematic review will generally vary with respect to design, study populations, and risk of bias [1].”
§ COSMOS-E: Guidance on conducting systematic reviews and meta-analyses of observational studies of etiology > Exploring and exploiting heterogeneity
What the sources say
- Partly supportedcites [1] MJ Page, 2016, Epidemiology and Reporting Characteristics of Systematic Reviews of Biomedical Research, A Cross-Sectional Study. PLoS Med sourceExperimental — accuracy not yet measured
The Results section on study eligibility criteria says that some categories of systematic reviews included a range of study designs. The Results section on risk-of-bias assessment reports that many reviews assessed the quality of included studies, and the Discussion warns about synthesizing clinically heterogeneous studies. Neither establishes that studies within a systematic review generally vary in all three specified respects—design, study populations, and risk of bias. (LLM, conf=0.81)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Subclinical or early disease might go undetected if such conditions are not ruled out explicitly.”
§ COSMOS-E: Guidance on conducting systematic reviews and meta-analyses of observational studies of etiology > Preparing the systematic review > Defining population, exposure, contrast, and outcome
hedgedstates a possible cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Reviewers may develop precise definitions and criteria for exposure and outcomes, but if no single study used these definitions, the review will not be able to answer the research question.”
§ COSMOS-E: Guidance on conducting systematic reviews and meta-analyses of observational studies of etiology > Preparing the systematic review > Defining population, exposure, contrast, and outcome
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
49 more claims
Create a free account to see all 168 claims and to verify the 94 references still open.
- Probably not a substantial claim. 91
- It may state two separate claims as one. 47
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“PRISMA 2020 is not intended to guide systematic review conduct, for which comprehensive resources are available.[43–46]”
From The PRISMA 2020 statement: An updated guideline for reporting systematic reviews (2021) · cites this paper as [44]
The cited source says what the claim attributes to it.
In the citing paper
“PRISMA 2020 is not intended to guide systematic review conduct, for which comprehensive resources are available [43–46].”
From The PRISMA 2020 statement: an updated guideline for reporting systematic reviews (2021) · cites this paper as [44]
In the citing paper
“El objetivo de la declaracio ´ n PRISMA 2020 no es orientar la realizacio ´ n de revisiones sistema ´ticas, para las que se dispone de varios recursos 42-45 .”
From Declaración PRISMA 2020: una guía actualizada para la publicación de revisiones sistemáticas (2021) · cites this paper as [43]
In the citing paper
“PRISMA 2020 is not intended to guide systematic review conduct, for which comprehensive resources are available [43-46] .”
From The PRISMA 2020 statement: An updated guideline for reporting systematic reviews (2021) · cites this paper as [44]
In the citing paper
“Case–control studies with less than ten matched-pairs were excluded due to the increased risk of bias (RoB) encountered in studies with sample sizes below this threshold”
From Risk-thresholds for the association between frequency of cannabis use and the development of psychosis: a systematic review and meta-analysis (2023) · cites this paper as Dekkers et al., 2019
1 more claim about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
Measuring inconsistency in meta-analyses
Higgins, J. P. T., Thompson, S. G., Deeks, J. J., & Altman, D. G. (2003) · BMJ · doi:10.1136/bmj.327.7414.557 · Cited by 63,139
Self-citationBias in meta-analysis detected by a simple, graphical test
Egger, M., Smith, G. D., Schneider, M., & Minder, C. (1997) · BMJ · doi:10.1136/bmj.315.7109.629 · Cited by 56,722
Self-citationThe Cochrane Collaboration's tool for assessing risk of bias in randomised trials
Higgins, J. P. T., Altman, D. G., Gotzsche, P. C., Juni, P., Moher, D., Oxman, A. D., Savovic, J., Schulz, K. F., Weeks, L., Sterne, J. A. C., et al. (2011) · BMJ · doi:10.1136/bmj.d5928 · Cited by 33,817
Self-citationLiberati, A., Altman, D. G., Tetzlaff, J., Mulrow, C., Gøtzsche, P. C., Ioannidis, J. P. A., Clarke, M., Devereaux, P. J., Kleijnen, J., & Moher, D. (2009) · PLoS Medicine · doi:10.1371/journal.pmed.1000100 · Cited by 28,164
Self-citationDermyshi, E., Wang, Y., Yan, C., Hong, W., Qiu, G., Gong, X., & Zhang, T. (2017) · Neonatology · doi:10.1159/000454668 · Cited by 24,451
Meta-analysis of Observational Studies in Epidemiology<SUBTITLE>A Proposal for Reporting</SUBTITLE>
Stroup, D. F. (2000) · JAMA · doi:10.1001/jama.283.15.2008 · Cited by 20,623
ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions
Sterne, J. A., Hernán, M. A., Reeves, B. C., Savović, J., Berkman, N. D., Viswanathan, M., Henry, D., Altman, D. G., Ansari, M. T., Boutron, I., et al. (2016) · BMJ · doi:10.1136/bmj.i4919 · Cited by 20,329
Self-citationStang, A. (2010) · European Journal of Epidemiology · doi:10.1007/s10654-010-9491-z · Cited by 17,396
QUADAS-2: a revised tool for the quality assessment of diagnostic accuracy studies
Whiting, P. F., Rutjes, A. W. S., Westwood, M. E., Mallett, S., Deeks, J. J., Reitsma, J. B., Leeflang, M. M. G., Sterne, J. A. C., Bossuyt, P. M. M., & the QUADAS-2 Group* (2011) · Annals of Internal Medicine · doi:10.7326/0003-4819-155-8-201110180-00009 · Cited by 14,199
Austin, P. C. (2011) · Multivariate Behavioral Research · doi:10.1080/00273171.2011.568786 · Cited by 12,242
Vandenbroucke, J. P., von Elm, E., Altman, D. G., Gøtzsche, P. C., Mulrow, C. D., Pocock, S. J., Poole, C., Schlesselman, J. J., Egger, M., & Initiative, F. T. S. (2007) · PLoS Medicine · doi:10.1371/journal.pmed.0040297 · Cited by 8,613
Self-citationGRADE guidelines: 3. Rating the quality of evidence
Balshem, H., Helfand, M., Schünemann, H. J., Oxman, A. D., Kunz, R., Brozek, J., Vist, G. E., Falck-Ytter, Y., Meerpohl, J., & Norris, S. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2010.07.015 · Cited by 8,285
Sterne, J. A. C., Sutton, A. J., Ioannidis, J. P. A., Terrin, N., Jones, D. R., Lau, J., Carpenter, J., Rucker, G., Harbord, R. M., Schmid, C. H., et al. (2011) · BMJ · doi:10.1136/bmj.d4002 · Cited by 7,709
Self-citationA basic introduction to fixed-effect and random-effects models for meta-analysis
Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. R. (2010) · Research Synthesis Methods · doi:10.1002/jrsm.12 · Cited by 6,793
Renehan, A. G., Tyson, M., Egger, M., Heller, R. F., & Zwahlen, M. (2008) · The Lancet · doi:10.1016/s0140-6736(08)60269-x · Cited by 5,219
Self-citationReading Mendelian randomisation studies: a guide, glossary, and checklist for clinicians
Davies, N. M., Holmes, M. V., & Davey Smith, G. (2018) · BMJ · doi:10.1136/bmj.k601 · Cited by 4,404
Funnel plots for detecting bias in meta-analysis
Sterne, J. A. C., & Egger, M. (2001) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(01)00377-8 · Cited by 3,779
Self-citationVandenbroucke, J. P., Elm, E. V., Altman, D. G., Gøtzsche, P. C., Mulrow, C. D., Pocock, S. J., Poole, C., Schlesselman, J. J., Egger, M., & initiative, F. T. S. (2007) · Annals of Internal Medicine · doi:10.7326/0003-4819-147-8-200710160-00010-w1 · Cited by 3,543
Self-citationSensitive questions in surveys.
Tourangeau, R., & Yan, T. (2007) · Psychological Bulletin · doi:10.1037/0033-2909.133.5.859 · Cited by 3,057
How should meta‐regression analyses be undertaken and interpreted?
Thompson, S. G., & Higgins, J. P. T. (2002) · Statistics in Medicine · doi:10.1002/sim.1187 · Cited by 3,026
A structural approach to selection bias
Hernán, M. A., Hernández-Díaz, S., & Robins, J. M. (2004) · Epidemiology · doi:10.1097/01.ede.0000135174.63482.43 · Cited by 2,844
Selective publication of antidepressant trials and its influence on apparent efficacy
Turner, E. H., Matthews, A. M., Linardatos, E., Tell, R. A., & Rosenthal, R. (2008) · New England Journal of Medicine · doi:10.1056/nejmsa065779 · Cited by 2,435
Methods for Trend Estimation from Summarized Dose-Response Data, with Applications to Meta-Analysis
Greenland, S., & Longnecker, M. P. (1992) · American Journal of Epidemiology · doi:10.1093/oxfordjournals.aje.a116237 · Cited by 2,353
Publication and related bias in meta-analysis
Sterne, J. A. C., Gavaghan, D., & Egger, M. (2000) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(00)00242-0 · Cited by 2,282
Self-citationGreenhalgh, T., & Peacock, R. (2005) · BMJ · doi:10.1136/bmj.38636.593461.68 · Cited by 2,045
The Hazards of Scoring the Quality of Clinical Trials for Meta-analysis
Jüni, P. (1999) · JAMA · doi:10.1001/jama.282.11.1054 · Cited by 1,981
Meta-analysis of individual participant data: rationale, conduct, and reporting
Riley, R. D., Lambert, P. C., & Abo-Zaid, G. (2010) · BMJ · doi:10.1136/bmj.c221 · Cited by 1,701
Renehan, A. G., Zwahlen, M., Minder, C., O'Dwyer, S. T., Shalet, S. M., & Egger, M. (2004) · The Lancet · doi:10.1016/s0140-6736(04)16044-3 · Cited by 1,697
Self-citationNegative Controls: A Tool for Detecting Confounding and Bias in Observational Studies
Lipsitch, M., Tchetgen Tchetgen, E., & Cohen, T. (2010) · Epidemiology · doi:10.1097/ede.0b013e3181d61eeb · Cited by 1,583
Sanderson, S., Tatt, I. D., & Higgins, J. P. (2007) · International Journal of Epidemiology · doi:10.1093/ije/dym018 · Cited by 1,531
The nuts and bolts of PROSPERO: an international prospective register of systematic reviews
Booth, A., Clarke, M., Dooley, G., Ghersi, D., Moher, D., Petticrew, M., & Stewart, L. (2012) · Systematic Reviews · doi:10.1186/2046-4053-1-2 · Cited by 1,433
The Extent and Consequences of P-Hacking in Science
Head, M. L., Holman, L., Lanfear, R., Kahn, A. T., & Jennions, M. D. (2015) · PLOS Biology · doi:10.1371/journal.pbio.1002106 · Cited by 1,388
Morgan, R. L., Whaley, P., Thayer, K. A., & Schünemann, H. J. (2018) · Environment International · doi:10.1016/j.envint.2018.07.015 · Cited by 1,282
Controlling the risk of spurious findings from meta-regression
Higgins, J. P. T., & Thompson, S. G. (2004) · Statistics in Medicine · doi:10.1002/sim.1752 · Cited by 1,267
Uncertainty in heterogeneity estimates in meta-analyses
Ioannidis, J. P. A., Patsopoulos, N. A., & Evangelou, E. (2007) · BMJ · doi:10.1136/bmj.39343.408449.80 · Cited by 1,254
Mendelian randomization: prospects, potentials, and limitations
Smith, G. D. (2004) · International Journal of Epidemiology · doi:10.1093/ije/dyh132 · Cited by 1,222
An introduction to instrumental variables for epidemiologists
Greenland, S. (2000) · International Journal of Epidemiology · doi:10.1093/ije/29.4.722 · Cited by 1,156
Triangulation in aetiological epidemiology
Lawlor, D. A., Tilling, K., & Davey Smith, G. (2017) · International Journal of Epidemiology · doi:10.1093/ije/dyw314 · Cited by 1,129
Meta-analysis Spurious precision? Meta-analysis of observational studies
Egger, M., Schneider, M., & Smith, G. D. (1998) · BMJ · doi:10.1136/bmj.316.7125.140 · Cited by 997
Self-citationInstruments for Causal Inference: An Epidemiologist's Dream?
Hernán, M. A., & Robins, J. M. (2006) · Epidemiology · doi:10.1097/01.ede.0000222409.00878.37 · Cited by 961
Page, M. J., Shamseer, L., Altman, D. G., Tetzlaff, J., Sampson, M., Tricco, A. C., Catalá-López, F., Li, L., Reid, E. K., Sarkis-Onofre, R., et al. (2016) · PLOS Medicine · doi:10.1371/journal.pmed.1002028 · Cited by 768
Self-citationHolmes, M. V., Dale, C. E., Zuccolo, L., Silverwood, R. J., Guo, Y., Ye, Z., Prieto-Merino, D., Dehghan, A., Trompet, S., Wong, A., et al. (2014) · BMJ · doi:10.1136/bmj.g4164 · Cited by 669
Adiposity and cancer risk: new mechanistic insights from epidemiology
Renehan, A. G., Zwahlen, M., & Egger, M. (2015) · Nature Reviews Cancer · doi:10.1038/nrc3967 · Cited by 654
Self-citationC Reactive Protein Coronary Heart Disease Genetics Collaboration (CCGC) (2011) · BMJ · doi:10.1136/bmj.d548 · Cited by 642
When are observational studies as credible as randomised trials?
Vandenbroucke, J. P. (2004) · The Lancet · doi:10.1016/s0140-6736(04)16261-2 · Cited by 538
Self-citationFeller, M., Huwiler, K., Stephan, R., Altpeter, E., Shang, A., Furrer, H., Pfyffer, G. E., Jemmi, T., Baumgartner, A., & Egger, M. (2007) · The Lancet Infectious Diseases · doi:10.1016/s1473-3099(07)70211-6 · Cited by 526
Self-citationSelf controlled case series methods: an alternative to standard epidemiological study designs
Petersen, I., Douglas, I., & Whitaker, H. (2016) · BMJ · doi:10.1136/bmj.i4515 · Cited by 525
Invited commentary: a critical look at some popular meta-analytic methods
Greenland, S. (1994) · American Journal of Epidemiology · doi:10.1093/oxfordjournals.aje.a117248 · Cited by 511
Male circumcision for prevention of heterosexual acquisition of HIV in men
Siegfried, N., Muller, M., Deeks, J. J., & Volmink, J. (2009) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd003362.pub2 · Cited by 494
Reasons or excuses for avoiding meta-analysis in forest plots
Ioannidis, J. P. A., Patsopoulos, N. A., & Rothstein, H. R. (2008) · BMJ · doi:10.1136/bmj.a117 · Cited by 457
Brain monoamine oxidase A inhibition in cigarette smokers
Fowler, J. S., Volkow, N. D., Wang, G.-J., Pappas, N., Logan, J., Shea, C., Alexoff, D., MacGregor, R. R., Schlyer, D. J., Zezulkova, I., et al. (1996) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.93.24.14065 · Cited by 423
Multisystem morbidity and mortality in Cushing’s syndrome: a cohort study
Dekkers, O. M., Horváth-Puhó, E., Jørgensen, J. O. L., Cannegieter, S. C., Ehrenstein, V., Vandenbroucke, J. P., Pereira, A. M., & Sørensen, H. T. (2013) · The Journal of Clinical Endocrinology & Metabolism · doi:10.1210/jc.2012-3582 · Cited by 419
Self-citationGast, K. B., Tjeerdema, N., Stijnen, T., Smit, J. W. A., & Dekkers, O. M. (2012) · PLoS ONE · doi:10.1371/journal.pone.0052036 · Cited by 403
Self-citationMueller, M., D’Addario, M., Egger, M., Cevallos, M., Dekkers, O., Mugglin, C., & Scott, P. (2018) · BMC Medical Research Methodology · doi:10.1186/s12874-018-0495-9 · Cited by 400
Self-citationDemonstration of deductive meta-analysis: ethanol intake and risk of myocardial infarction
Maclure, M. (1993) · Epidemiologic Reviews · doi:10.1093/oxfordjournals.epirev.a036124 · Cited by 397
Does obesity shorten life? The importance of well-defined interventions to answer causal questions
Hernán, M. A., & Taubman, S. L. (2008) · International Journal of Obesity · doi:10.1038/ijo.2008.82 · Cited by 384
Evaluation of old and new tests of heterogeneity in epidemiologic meta-analysis
Takkouche, B., Cadarso-Suarez, C., & Spiegelman, D. (1999) · American Journal of Epidemiology · doi:10.1093/oxfordjournals.aje.a009981 · Cited by 380
Education and myopia: assessing the direction of causality by mendelian randomisation
Mountjoy, E., Davies, N. M., Plotnikov, D., Smith, G. D., Rodriguez, S., Williams, C. E., Guggenheim, J. A., & Atan, D. (2018) · BMJ · doi:10.1136/bmj.k2022 · Cited by 299
Different Patterns of Duplicate Publication: An Analysis of Articles Used in Systematic Reviews
von Elm, E., Poglia, G., Walder, B., & Tramèr, M. R. (2004) · JAMA · doi:10.1001/jama.291.8.974 · Cited by 290
Biases in Randomized Trials: A Conversation Between Trialists and Epidemiologists
Mansournia, M. A., Higgins, J. P. T., Sterne, J. A. C., & Hernán, M. A. (2017) · Epidemiology · doi:10.1097/ede.0000000000000564 · Cited by 285
GRADE: Assessing the quality of evidence in environmental and occupational health
Morgan, R. L., Thayer, K. A., Bero, L., Bruce, N., Falck-Ytter, Y., Ghersi, D., Guyatt, G., Hooijmans, C., Langendam, M., Mandrioli, D., et al. (2016) · Environment International · doi:10.1016/j.envint.2016.01.004 · Cited by 277
Systematic reviews need systematic searchers.
(2005) · Europe PMC (PubMed Central) · Cited by 257
Case-control studies: basic concepts
Vandenbroucke, J. P., & Pearce, N. (2012) · International Journal of Epidemiology · doi:10.1093/ije/dys147 · Cited by 254
Self-citationBlair, A., Stewart, P., Lubin, J. H., & Forastiere, F. (2007) · American Journal of Industrial Medicine · doi:10.1002/ajim.20281 · Cited by 251
Data Extraction Errors in Meta-analyses That Use Standardized Mean Differences
Gøtzsche, P. C., Hróbjartsson, A., Marić, K., & Tendal, B. (2007) · JAMA · doi:10.1001/jama.298.4.430 · Cited by 238
Breastfeeding and Childhood Leukemia Incidence: A Meta-analysis and Systematic Review
Amitay, E. L., & Keinan-Boker, L. (2015) · JAMA Pediatrics · doi:10.1001/jamapediatrics.2015.1025 · Cited by 234
Graphical displays for meta-analysis: An overview with suggestions for practice
Anzures‐Cabrera, J., & Higgins, J. P. T. (2010) · Research Synthesis Methods · doi:10.1002/jrsm.6 · Cited by 233
Hartemink, N., Boshuizen, H. C., Nagelkerke, N. J. D., Jacobs, M. A. M., & van Houwelingen, H. C. (2006) · American Journal of Epidemiology · doi:10.1093/aje/kwj141 · Cited by 226
Geographical Patterns of Male Circumcision Practices in Africa: Association with HIV Seroprevalence
MOSES, S., BRADLEY, J. E., NAGELKERKE, N. J. D., RONALD, A. R., NDINYA-ACHOLA, J. O., & PLUMMER, F. A. (1990) · International Journal of Epidemiology · doi:10.1093/ije/19.3.693 · Cited by 223
How to assess the external validity of therapeutic trials: a conceptual approach
Dekkers, O. M., Elm, E. V., Algra, A., Romijn, J. A., & Vandenbroucke, J. P. (2010) · International Journal of Epidemiology · doi:10.1093/ije/dyp174 · Cited by 219
Self-citationInvited commentary: benefits of heterogeneity in meta-analysis of data from epidemiologic studies
Berlin, J. A. (1995) · American Journal of Epidemiology · doi:10.1093/oxfordjournals.aje.a117645 · Cited by 191
Knol, M. J., Vandenbroucke, J. P., Scott, P., & Egger, M. (2008) · American Journal of Epidemiology · doi:10.1093/aje/kwn217 · Cited by 189
Self-citationCase-control matching: effects, misconceptions, and recommendations
Mansournia, M. A., Jewell, N. P., & Greenland, S. (2018) · European Journal of Epidemiology · doi:10.1007/s10654-017-0325-0 · Cited by 175
Meta-analysis. Beyond the grand mean?
Smith, G. D., Egger, M., & Phillips, A. N. (1997) · BMJ · doi:10.1136/bmj.315.7122.1610 · Cited by 174
Self-citationRassen, J. A., Brookhart, M. A., Glynn, R. J., Mittleman, M. A., & Schneeweiss, S. (2009) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2008.12.005 · Cited by 164
Mobile phone use and brain tumours in the CERENAT case-control study
Coureau, G., Bouvier, G., Lebailly, P., Fabbro-Peray, P., Gruber, A., Leffondre, K., Guillamo, J.-S., Loiseau, H., Mathoulin-Pélissier, S., Salamon, R., et al. (2014) · Occupational and Environmental Medicine · doi:10.1136/oemed-2013-101754 · Cited by 159
Davey Smith, G., Phillips, A. N., & Neaton, J. D. (1992) · The Lancet · doi:10.1016/0140-6736(92)92242-8 · Cited by 149
Burgers, A. M. G., Biermasz, N. R., Schoones, J. W., Pereira, A. M., Renehan, A. G., Zwahlen, M., Egger, M., & Dekkers, O. M. (2011) · The Journal of Clinical Endocrinology & Metabolism · doi:10.1210/jc.2011-1377 · Cited by 149
Self-citationda Costa, B. R., Rutjes, A. W., Johnston, B. C., Reichenbach, S., Nüesch, E., Tonia, T., Gemperli, A., Guyatt, G. H., & Jüni, P. (2012) · International Journal of Epidemiology · doi:10.1093/ije/dys124 · Cited by 145
Searching one or two databases was insufficient for meta-analysis of observational studies
Lemeshow, A. R., Blum, R. E., Berlin, J. A., Stoto, M. A., & Colditz, G. A. (2005) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2005.03.004 · Cited by 137
Mason, K., Pearce, N. E., & Cummins, S. (2017) · The Lancet Public Health · doi:10.1016/s2468-2667(17)30212-8 · Cited by 127
Morgan, R. L., Thayer, K. A., Santesso, N., Holloway, A. C., Blain, R., Eftim, S. E., Goldstone, A. E., Ross, P., Guyatt, G., & Schünemann, H. J. (2018) · Environment International · doi:10.1016/j.envint.2018.08.018 · Cited by 106
Gielen, C., Dekkers, O., Stijnen, T., Schoones, J., Brand, A., Klautz, R., & Eikenboom, J. (2014) · Interactive CardioVascular and Thoracic Surgery · doi:10.1093/icvts/ivt506 · Cited by 104
Self-citationRoest, A. M., de Jonge, P., Williams, C. D., de Vries, Y. A., Schoevers, R. A., & Turner, E. H. (2015) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2015.15 · Cited by 101
Ponjoan, A., Blanch, J., Alves‐Cabratosa, L., Martí‐Lluch, R., Comas‐Cufí, M., Parramon, D., García-Gil, M., Ramos, R., & Petersen, I. (2017) · Environmental Health · doi:10.1186/s12940-017-0238-0 · Cited by 93
PEDro's bias: summary quality scores should not be used in meta-analysis
da Costa, B. R., Hilfiker, R., & Egger, M. (2013) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2012.08.003 · Cited by 86
Self-citationA primer on systematic reviews in toxicology
Hoffmann, S., de Vries, R. B. M., Stephens, M. L., Beck, N. B., Dirven, H. A. A. M., Fowle, J. R., Goodman, J. E., Hartung, T., Kimber, I., Lalu, M. M., et al. (2017) · Archives of Toxicology · doi:10.1007/s00204-017-1980-3 · Cited by 85
Modern Meta-Analysis: Review and Update of Methodologies
Cleophas, T. J., & Zwinderman, A. H. (2017) · doi:10.1007/978-3-319-55895-0 · Cited by 83
Tendal, B., Higgins, J. P. T., Juni, P., Hrobjartsson, A., Trelle, S., Nuesch, E., Wandel, S., Jorgensen, A. W., Gesser, K., Ilsoe-Kristensen, S., et al. (2009) · BMJ · doi:10.1136/bmj.b3128 · Cited by 77
Epidemiologic measures and policy formulation: lessons from potential outcomes
Greenland, S. (2005) · Emerging Themes in Epidemiology · doi:10.1186/1742-7622-2-5 · Cited by 76
Costanza, M. C. (1995) · Preventive Medicine · doi:10.1006/pmed.1995.1069 · Cited by 56
Selective association of multiple sclerosis with infectious mononucleosis
Zaadstra, B., Chorus, A., van Buuren, S., Kalsbeek, H., & van Noort, J. (2008) · Multiple Sclerosis Journal · doi:10.1177/1352458507084265 · Cited by 41
Mendelian randomization studies in the elderly
Boef, A. G. C., le Cessie, S., & Dekkers, O. M. (2015) · Epidemiology · doi:10.1097/ede.0000000000000243 · Cited by 39
Self-citationKuper, H., Nicholson, A., & Hemingway, H. (2006) · BMC Medical Research Methodology · doi:10.1186/1471-2288-6-4 · Cited by 36
“Case-Control” Confusion: Mislabeled Reports in Obstetrics and Gynecology Journals
Grimes, D. A. (2009) · Obstetrics & Gynecology · doi:10.1097/aog.0b013e3181c03421 · Cited by 33
Cooper, G. S., Lunn, R. M., Ågerstrand, M., Glenn, B. S., Kraft, A. D., Luke, A. M., & Ratcliffe, J. M. (2016) · Environment International · doi:10.1016/j.envint.2016.03.017 · Cited by 33
Ford, N. P., Mills, E. J., & Egger, M. (2014) · Clinical Infectious Diseases · doi:10.1093/cid/ciu1138 · Cited by 31
Self-citationAlthuis, M. D., Weed, D. L., & Frankenfeld, C. L. (2014) · Systematic Reviews · doi:10.1186/2046-4053-3-80 · Cited by 30
Case-control studies in neurosurgery: A review
Nesvick, C. L., Thompson, C. J., Boop, F. A., & Klimo, P. (2014) · Journal of Neurosurgery · doi:10.3171/2014.5.jns132329 · Cited by 22
Bartholomew, L. (1998) · doi:10.1097/00006254-199811000-00024 · Cited by 18
Case–control study of lifetime alcohol consumption and endometrial cancer risk
Friedenreich, C. M., Speidel, T. P., Neilson, H. K., Langley, A. R., Courneya, K. S., Magliocco, A. M., & Cook, L. S. (2013) · Cancer Causes & Control · doi:10.1007/s10552-013-0275-0 · Cited by 12
Waffenschmidt, S., Hermanns, T., Gerber-Grote, A., & Mostardt, S. (2017) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2016.08.008 · Cited by 11
Commentary: 'Smoking and lung cancer'--the embryogenesis of modern epidemiology
Vandenbroucke, J. P. (2009) · International Journal of Epidemiology · doi:10.1093/ije/dyp292 · Cited by 10
Self-citationTransforming Odds Ratios into Correlation Coefficients
Cleophas, T. J., & Zwinderman, A. H. (2017) · Springer eBooks · doi:10.1007/978-3-319-55895-0_20 · Cited by 6
A simple method for converting an odds ratio to effect size for use in meta-analysis
Chinn, S. (2000) · Statistics in Medicine · doi:10.1002/1097-0258(20001130)19:22<3127::aid-sim784>3.0.co;2-m
Cited by
The PRISMA 2020 statement: An updated guideline for reporting systematic reviews
Page, M. J., McKenzie, J. E., Bossuyt, P. M., Boutron, I., Hoffmann, T. C., Mulrow, C. D., Shamseer, L., Tetzlaff, J. M., Akl, E. A., Brennan, S. E., et al. (2021) · BMJ · doi:10.1136/bmj.n71 · Cited by 108,283
The PRISMA 2020 statement: an updated guideline for reporting systematic reviews
Page, M. J., McKenzie, J. E., Bossuyt, P. M., Boutron, I., Hoffmann, T. C., Mulrow, C. D., Shamseer, L., Tetzlaff, J. M., Akl, E. A., Brennan, S. E., et al. (2021) · Systematic Reviews · doi:10.1186/s13643-021-01626-4 · Cited by 13,874
The PRISMA 2020 Statement: An Updated Guideline for Reporting Systematic Reviews
Page, M. J., McKenzie, J. E., Bossuyt, P. M., Boutron, I., Hoffmann, T. C., Mulrow, C. D., Shamseer, L., Tetzlaff, J. M., Akl, E. A., Brennan, S. E., et al. (2021) · International Journal of Surgery · doi:10.1016/j.ijsu.2021.105906 · Cited by 11,748
The PRISMA 2020 statement: an updated guideline for reporting systematic reviews
Page, M. J., McKenzie, J., Bossuyt, P., Boutron, I., Hoffmann, T., mulrow, C., Shamseer, L., Tetzlaff, J., Akl, E., Brennan, S. E., et al. (2020) · doi:10.31222/osf.io/v7gm2 · Cited by 5,177
The PRISMA 2020 statement: An updated guideline for reporting systematic reviews
Page, M. J., McKenzie, J. E., Bossuyt, P. M., Boutron, I., Hoffmann, T. C., Mulrow, C. D., Shamseer, L., Tetzlaff, J. M., Akl, E. A., Brennan, S. E., et al. (2021) · PLoS Medicine · doi:10.1371/journal.pmed.1003583 · Cited by 5,148
Metadata
- Authors
- Dekkers, O. M., Vandenbroucke, J. P., Cevallos, M., Renehan, A. G., Altman, D. G., & Egger, M.
- Affiliations
- Leiden University Medical Center (NL)
- University of Bern (CH)
- University of Manchester (GB)
- University of Oxford (GB)
- Corresponding author
- Dekkers, Olaf M.
- ORCID
- Venue
- PLOS Medicine
- Publisher
- Public Library of Science (PLoS)
- Published
- 2019-02-21
- Volume
- 16
- Issue
- 2
- Pages
- e1002742
- ISSN
- 1549-1277 (ISSN-L)
- 1549-1676
- PMC
- PMC6383865
- OpenAlex
- W2917632902
- Semantic Scholar
- 4fda707b38de63950725cb84e6342baa4ffd57e7
- Cited by
- 525
- References
- 105
- Citations by year
- 2026: 45 · 2025: 78 · 2024: 101 · 2023: 79 · 2022: 63 · 2021: 93 · 2020: 53 · 2019: 13
- FWCI
- 11.25
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- PLoS Medicine (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Data Archiving and Networked Services (DANS) (submittedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- Leiden Repository (Leiden University) (submittedVersion)
- Bern Open Repository and Information System (University of Bern) (submittedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Figshare (submittedVersion, cc-by)
- Funders
- National Science Foundation
- Sanofi
- Cancer Research UK (16895)
- Schweizerischer Nationalfonds zur Förderung der Wissenschaftlichen Forschung (174281)
- Novo Nordisk
- Sanofi Pasteur
- Cilag
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Physical Sciences › Mathematics › Statistics and Probability › Advanced Causal Inference Techniques
- Keywords
- meta-analysis, risk of bias assessment, confounding, selection bias, information bias, heterogeneity, causality, research question formulation
- MeSH terms
- Systematic Reviews as Topic, Humans, Meta-Analysis as Topic, Selection Bias, Observational Studies as Topic
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-04, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-04 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol and qwen3.8-omni-flash).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.