The DOI matches its Crossref record; 104 of the 104 references it cites were found in a registry or an index; 48 of the 374 sources its claims cite were checked for support.
Methods to systematically review and meta-analyse observational studies: a systematic scoping review of recommendations
Open accessAbstract
BACKGROUND: Systematic reviews and meta-analyses of observational studies are frequently performed, but no widely accepted guidance is available at present. We performed a systematic scoping review of published methodological recommendations on how to systematically review and meta-analyse observational studies. METHODS: We searched online databases and websites and contacted experts in the field to locate potentially eligible articles. We included articles that provided any type of recommendation on how to conduct systematic reviews and meta-analyses of observational studies. We extracted and summarised recommendations on pre-defined key items: protocol development, research question, search strategy, study eligibility, data extraction, dealing with different study designs, risk of bias assessment, publication bias, heterogeneity, statistical analysis. We summarised recommendations by key item, identifying areas of agreement and disagreement as well as areas where recommendations were missing or scarce. RESULTS: The searches identified 2461 articles of which 93 were eligible. Many recommendations for reviews and meta-analyses of observational studies were transferred from guidance developed for reviews and meta-analyses of RCTs. Although there was substantial agreement in some methodological areas there was also considerable disagreement on how evidence synthesis of observational studies should be conducted. Conflicting recommendations were seen on topics such as the inclusion of different study designs in systematic reviews and meta-analyses, the use of quality scales to assess the risk of bias, and the choice of model (e.g. fixed vs. random effects) for meta-analysis. CONCLUSION: There is a need for sound methodological guidance on how to conduct systematic reviews and meta-analyses of observational studies, which critically considers areas in which there are conflicting recommendations.
Overview
CoChat Summary
TL;DR
This systematic scoping review identified 93 articles giving methodological recommendations for systematic reviews and meta-analyses of observational studies. It found many recommendations were transferred from RCT guidance and that substantial disagreement remains on key methodological choices, with no comprehensive guidance available.
Key findings
- The searches identified 2461 articles; 93 were eligible. Only 1 (1%) addressed all 10 key items, 56 (60%) addressed two or more key items, and 37 (40%) addressed only one key item.
- Conflicting recommendations were found for “research question”, “study eligibility”, “considering different study designs”, “risk of bias assessment”, “publication bias” and “statistical analysis”. Examples include narrow versus broad research questions, whether to include all languages, whether to combine RCTs and non-randomised studies in one meta-analysis, use of quality scales versus component approaches, and fixed- versus random-effects models.
- Most included articles (62%) were published in statistical, epidemiological or systematic review methodological journals; 72% were written by authors affiliated with a systematic review organisation or a statistical/epidemiological university department. Many recommendations for observational studies were uncritically transferred from guidance for RCTs.
- Areas of agreement included the importance of a protocol, systematic risk-of-bias assessment, and examination of heterogeneity as central to meta-analysis of observational studies.
Methods
- Systematic scoping review following a protocol written before starting. Searches covered Medline, the Cochrane Methodology Register, websites of review centres, selected evidence-synthesis journals, expert contact, and references from included full texts; no language restrictions were applied. Initial search was January 2014, repeated February 2017 except for CMR.
- Included articles published from 1994 onward that provided recommendations on at least one of 10 a priori key methodological items. Excluded articles published before 1994, reviews of observational studies without methodological recommendations, and articles focused on RCTs, cost-effectiveness studies or diagnostic studies.
- Two reviewers independently screened and extracted data; disagreements were resolved by discussion or a senior methodologist. Recommendations were summarised descriptively and by qualitative thematic analysis. The authors did not appraise the methodological quality of included articles or recommendations.
Limitations
- Stated: including articles based on title and abstract may have missed relevant articles; books were excluded, including the Cochrane handbook; the authors did not assess the soundness of recommendations; and in-depth discussion of statistical methods was beyond scope.
- Stated: the authors did not appraise methodological quality of included articles and recommendations because widely accepted standards for systematic reviews and meta-analyses of observational studies are lacking.
- Evident from the text: the search relied on theoretical saturation rather than exhaustive retrieval, and the last search update was February 2017, so later guidance would not be captured.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
18 Valid · 2 Not valid · 67 Not checked · 64 Own results and interpretations
At least one cited source doesn’t support this claim or points to a different paper, and none confirmed it — see the verdicts below.
In the paper
“Second, the use of a random-effects model for meta-analysis of observational studies by default was frequently recommended [8, 16, 29, 34, 70, 77, 100, 101].”
§ Methods to systematically review and meta-analyse observational studies: a systematic scoping review of recommendations > Results > Statistical analysis
What the sources say
- Not checked yetcites [8] Zwahlen M, Renehan A, Egger M. Meta-analysis in medical research: potentials and limitations. Urol Oncol. 2008;26:320–9. source
- Not checked yetcites [16] Lau J, Ioannidis JP, Schmid CH. Quantitative synthesis in systematic reviews. Ann Intern Med. 1997;127:820–6. source
- Not checked yetcites [29] Egger M, Smith GD, Phillips AN. Meta-analysis: principles and procedures. BMJ. 1997;315:1533–7. source
- Not checked yetcites [34] Blair A, Burg J, Foran J, Gibb H, Greenland S, Morris R, et al. Guidelines for application of meta-analysis in environmental epidemiology. ISLI risk science institute. Regul Toxicol Pharmacol. 1995;22:189–97. source
- Disputedcites [70] Wong O, Raabe GK. Application of meta-analysis in reviewing occupational cohort studies. Occup Environ Med. 1996;53:793–800. sourceExperimental — accuracy not yet measured
In the discussion of fixed- and random-effects models (p. 5), the paper says the choice is subject to ongoing debate, that commonly used methods are fixed-effects, and that random-effects models have been criticised. It does not recommend using random-effects models by default for observational studies; its account points in the opposite direction. (LLM, conf=0.85)
Checked by a reasoning model — may be wrong.
- Not checked yetcites [77] Colditz GA, Burdick E, Mosteller F. Heterogeneity in meta-analysis of data from epidemiologic studies: a commentary. Am J Epidemiol. 1995;142:371–82. source
- Not checked yetcites [100] Pladevall-Vila M, Delclos GL, Varas C, Guyer H, Brugues-Tarradellas J, Anglada-Arisa A. Controversy of oral contraceptives and risk of rheumatoid arthritis: meta-analysis of conflicting studies and review of conflicting meta-analyses with special emphasis on analysis of heterogeneity. Am J Epidemiol. 1996;144:1–14. source
- Not checked yetcites [101] Dwyer T, Couper D, Walter SD. Sources of heterogeneity in the meta-analysis of observational studies: the example of SIDS and sleeping position. J Clin Epidemiol. 2001;54:440–7. source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The sources with a verdict support this claim; the rest weren’t checked yet, have no full text we could read, or their verdicts aren’t rated yet.
In the paper
“Aspects that were frequently mentioned as important to address were the population, exposures or interventions, and outcomes [38–41, 43, 47, 48, 50, 54].”
§ Methods to systematically review and meta-analyse observational studies: a systematic scoping review of recommendations > Results > Research question
What the sources say
- Supportedcites [38] Simunovic N, Sprague S, Bhandari M. Methodological issues in systematic reviews and meta-analyses of observational studies in orthopaedic research. J Bone Jt Surg Am. 2009;91(Suppl 3):87–94. source
The Introduction says that a systematic-review protocol should be centered on a focused research question and clearly describe the population, intervention or exposure, and outcomes of interest. This supports the aspects attributed to this source; the paper does not itself establish how frequently other sources mention them. (LLM, conf=0.92)
- Not checked yetcites [39] Armstrong R, Waters E, editors. Systematic Reviews of Health Promotion and Public Health Interventions. Guidelines for Systematic Reviews in Health Promotion and Public Health Task Force. Version 2. Melbourne University: Australia.; 2007.
- Supportedcites [40] Schünemann HJ, Tugwell P, Reeves BC, Akl EA, Santesso N, Spencer FA, et al. Non-randomized studies as a source of complementary, sequential or replacement evidence for randomized controlled trials in systematic reviews on the effects of interventions. Res Synth Methods. 2013;4:49–62. source
Section 3.1 (p. 7) says review authors should define the elements of their PICO question when planning a review. Section 1.2 (p. 2) explicitly identifies population, intervention and outcomes among the elements to consider. The paper supports the cited point that these aspects are important to address, though it does not itself measure how frequently they are mentioned across sources. (LLM, conf=0.88)
- Not checked yetcites [41] Moola S, Munn Z, Sears K, Sfetcu R, Currie M, Lisy K, et al. Conducting systematic reviews of association (etiology): the Joanna Briggs Institute’s approach. Int J Evid Based Healthc. 2015;13:163–9. source
- Supportedcites [43] Ravani P, Ronksley PE, James MT, Strippoli GF. A nephrology guide to reading and using systematic reviews of observational studies. Nephrol Dial Transplant. 2015;30:1615–21. source
Step 1 (p. 3) says that a systematic review’s question is specified using PICO and identifies population, intervention or exposure, and outcome as components. The source also includes a comparator, but the citing sentence does not claim its list is exhaustive. (LLM, conf=0.92)
- Not checked yetcites [47] Manchikanti L, Datta S, Smith HS, Hirsch JA. Evidence-based medicine, systematic reviews, and guidelines in interventional pain management: part 6. Systematic reviews and meta-analyses of observational studies. Pain Physician. 2009;12:819–50. source
- Not checked yetcites [48] Normand SL. Meta-analysis: formulating, evaluating, combining, and reporting. Stat Med. 1999;18:321–59. source
- Not checked yetcites [50] Raman G, Gaylor JM, Rao M, Chan J, Earley A, Chang LKW, et al. AHRQ methods for effective health care. In: Quality of reporting in systematic reviews of implantable medical devices. Rockville (MD): Agency for Healthcare Research and Quality (US); 2012.
- Not checked yetcites [54] Chaiyakunapruk N, Saokaew S, Sruamsiri R, Dilokthornsakul P. Systematic review and network meta-analysis in health technology assessment. J Med Assoc Thail. 2014;97(Suppl 5):S33–42.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The sources with a verdict support this claim; the rest weren’t checked yet, have no full text we could read, or their verdicts aren’t rated yet.
In the paper
“Only five articles referred to the combination of different non-randomised studies [38, 70, 75, 79, 80].”
§ Methods to systematically review and meta-analyse observational studies: a systematic scoping review of recommendations > Results > Considering different study designs
What the sources say
- Supportedcites [38] Simunovic N, Sprague S, Bhandari M. Methodological issues in systematic reviews and meta-analyses of observational studies in orthopaedic research. J Bone Jt Surg Am. 2009;91(Suppl 3):87–94. source
The Introduction discusses whether different observational study designs should be combined and says that different study types should rarely be merged in a meta-analysis. This supports citing the paper as one article that refers to combining non-randomised studies; the count of five describes the citing document’s set of references, not a finding attributed to this paper. (LLM, conf=0.91)
- Supportedcites [70] Wong O, Raabe GK. Application of meta-analysis in reviewing occupational cohort studies. Occup Environ Med. 1996;53:793–800. source
The Meta-review of occupational epidemiology (pp. 2–4) discusses when different occupational cohort studies can be combined in a meta-analysis. This establishes the cited paper as an example of an article addressing the combination of non-randomised studies; the count of five refers to the citing paper’s set of references, not a count this source needs to establish. (LLM, conf=0.91)
- Not checked yetcites [75] Valentine JC, Thompson SG. Issues relating to confounding and meta-analysis when including non-randomized studies in systematic reviews on the effects of interventions. Res Synth Methods. 2013;4:26–35. source
- Not checked yetcites [79] Martin DO, Austin H. An exact method for meta-analysis of case-control and follow-up studies. Epidemiology. 2000;11:255–60. source
- Not checked yetcites [80] Moreno V, Martin ML, Bosch FX, de Sanjose S, Torres F, Munoz N. Combined analysis of matched and unmatched case-control studies: comparison of risk estimates from different studies. Am J Epidemiol. 1996;143:293–300. source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The sources with a verdict support this claim; the rest weren’t checked yet, have no full text we could read, or their verdicts aren’t rated yet.
In the paper
“Most articles considered the combination of RCTs and non-randomised designs [29, 35, 38, 39, 49, 51, 56, 61, 75, 81].”
§ Methods to systematically review and meta-analyse observational studies: a systematic scoping review of recommendations > Results > Considering different study designs
What the sources say
- Not checked yetcites [29] Egger M, Smith GD, Phillips AN. Meta-analysis: principles and procedures. BMJ. 1997;315:1533–7. source
- Supportedcites [35] Loke YK, Price D, Herxheimer A. Systematic reviews of adverse effects: framework for a structured approach. BMC Med Res Methodol. 2007;7:32. source
The Methods section (‘What types of studies?’) considers RCTs alongside cohort, case-control and other non-randomised designs for reviews of adverse effects. This paper supports its inclusion as one of the cited articles that considered both types of design; it does not itself establish the collective claim that most articles did so. (LLM, conf=0.88)
- Supportedcites [38] Simunovic N, Sprague S, Bhandari M. Methodological issues in systematic reviews and meta-analyses of observational studies in orthopaedic research. J Bone Jt Surg Am. 2009;91(Suppl 3):87–94. source
The Introduction considers whether randomized controlled trials and observational studies should be included in the same review. As one of several citations supporting the citing authors’ summary of the articles, this paper is an example of an article that considers that combination. (LLM, conf=0.88)
- Not checked yetcites [39] Armstrong R, Waters E, editors. Systematic Reviews of Health Promotion and Public Health Interventions. Guidelines for Systematic Reviews in Health Promotion and Public Health Task Force. Version 2. Melbourne University: Australia.; 2007.
- Not checked yetcites [49] Price D, Jefferson T, Demicheli V. Methodological issues arising from systematic reviews of the evidence of safety of vaccines. Vaccine. 2004;22:2080–4. source
- Not checked yetcites [51] Rosenthal R, DiMatteo MR. Meta-analysis: recent developments in quantitative methods for literature reviews. Annu Rev Psychol. 2001;52:59–82. source
- Not checked yetcites [56] Wille-Jorgensen P, Renehan AG. Systematic reviews and meta-analyses in coloproctology: interpretation and potential pitfalls. Color Dis. 2008;10:21–32. source
- Not checked yetcites [61] Thornton A, Lee P. Publication bias in meta-analysis: its causes and consequences. J Clin Epidemiol. 2000;53:207–16. source
- Not checked yetcites [75] Valentine JC, Thompson SG. Issues relating to confounding and meta-analysis when including non-randomized studies in systematic reviews on the effects of interventions. Res Synth Methods. 2013;4:26–35. source
- Supportedcites [81] Souverein OW, Dullemeijer C, van’t Veer P, van der Voet H. Transformations of summary statistics as input in meta-analysis for linear dose-response models on a logarithmic scale: a methodology developed within EURRECA. BMC Med Res Methodol. 2012;12:57. source
The Background explicitly considers combining observational studies and RCTs in a single meta-analysis. The Discussion says the methodology makes their results comparable and identifies combining them as a possible application. This supports citing the paper as an article that considered the combination; the count of 'most articles' is the citing authors' synthesis, not a claim about this individual source. (LLM, conf=0.93)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Every source the paper cites for this claim was checked and supports it.
In the paper
“However, our search was guided by the methodological principle of theoretical saturation [17, 18].”
§ Methods to systematically review and meta-analyse observational studies: a systematic scoping review of recommendations > Discussion
What the sources say
- Supportedcites [17] Lilford RJ, Richardson A, Stevens A, Fitzpatrick R, Edwards S, Rock F, et al. Issues in methodological research: perspectives from researchers and commissioners. Health Technol Assess. 2001;5:1–57. source
In “Reflections on the findings,” the paper defines theoretical saturation as the point at which further searching yields no new insights and endorses systematicity rather than exhaustiveness. This supports its use as a principle guiding when to stop a literature search; the citing authors’ description of their own search is not a finding attributed to the source. (LLM, conf=0.92)
- Supportedcites [18] Rutjes A, Reitsma J, Coomarasamy A, Khan K, Bossuyt P. Evaluation of diagnostic tests when there is no gold standard. A review of methods. Health Technol Assess. 2007;11(50). source
The “Literature search and inclusion of papers” section explicitly identifies theoretical saturation as a principle that guided the source paper’s search and selection. This supports citing it as a methodological precedent; the source does not independently establish how the citing authors conducted their own search. (LLM, conf=0.94)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
43 more claims · 1 with their cited sources checked
Create a free account to see all 151 claims and to verify the 326 references still open.
- Probably not a substantial claim. 81
- It may state two separate claims as one. 45
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“There is no standard answer to the question of whether study design or methodological quality should guide inclusion of studies [18].”
From COSMOS-E: Guidance on conducting systematic reviews and meta-analyses of observational studies of etiology (2019) · cites this paper as [18]
In the citing paper
“and some design issues are not settled yet [18].”
From COSMOS-E: Guidance on conducting systematic reviews and meta-analyses of observational studies of etiology (2019) · cites this paper as [18]
In the citing paper
“Authors provide different reasons for not pooling data [69], and different opinions exist on how to approach this question [18].”
From COSMOS-E: Guidance on conducting systematic reviews and meta-analyses of observational studies of etiology (2019) · cites this paper as [18]
In the citing paper
“The assumption that all these studies estimate the same underlying effect is rarely justified, and using a random-effects model for combining observational studies therefore seems reasonable [18].”
From COSMOS-E: Guidance on conducting systematic reviews and meta-analyses of observational studies of etiology (2019) · cites this paper as [18]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Methods to systematically review and meta-analyse observational studies: a systematic scoping review of recommendations > Results > Overview of recommendations
“| Egger, 1997b [97] | Observational | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✓ | ✗ | ✗ |”
§ Methods to systematically review and meta-analyse observational studies: a systematic scoping review of recommendations > References
“97. Egger M Smith GD Bias in location and selection of studies BMJ 1998 316 61 66 10.1136/bmj.316.7124.61 9451274 PMC2665334”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Methods to systematically review and meta-analyse observational studies: a systematic scoping review of recommendations > Results > Study eligibility
“Five articles gave recommendations about how to assess eligibility: it should be blind [29, 30, 46], independent [29, 33, 46, 48], and performed by two reviewers [29, 30, 33, 48].”
§ Methods to systematically review and meta-analyse observational studies: a systematic scoping review of recommendations > Results > Overview of recommendations
“| | | Assessment of eligibility | 6 (27%) |”
Citations
Cites
Liberati, A., Altman, D. G., Tetzlaff, J., Mulrow, C., Gøtzsche, P. C., Ioannidis, J. P. A., Clarke, M., Devereaux, P. J., Kleijnen, J., & Moher, D. (2009) · PLoS Medicine · doi:10.1371/journal.pmed.1000100 · Cited by 28,164
Dermyshi, E., Wang, Y., Yan, C., Hong, W., Qiu, G., Gong, X., & Zhang, T. (2017) · Neonatology · doi:10.1159/000454668 · Cited by 24,451
Meta-analysis of Observational Studies in Epidemiology<SUBTITLE>A Proposal for Reporting</SUBTITLE>
Stroup, D. F. (2000) · JAMA · doi:10.1001/jama.283.15.2008 · Cited by 20,623
ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions
Sterne, J. A., Hernán, M. A., Reeves, B. C., Savović, J., Berkman, N. D., Viswanathan, M., Henry, D., Altman, D. G., Ansari, M. T., Boutron, I., et al. (2016) · BMJ · doi:10.1136/bmj.i4919 · Cited by 20,329
Liberati, A., Altman, D. G., Tetzlaff, J., Mulrow, C., Gotzsche, P. C., Ioannidis, J. P. A., Clarke, M., Devereaux, P. J., Kleijnen, J., & Moher, D. (2009) · BMJ · doi:10.1136/bmj.b2700 · Cited by 17,648
GRADE guidelines: 3. Rating the quality of evidence
Balshem, H., Helfand, M., Schünemann, H. J., Oxman, A. D., Kunz, R., Brozek, J., Vist, G. E., Falck-Ytter, Y., Meerpohl, J., & Norris, S. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2010.07.015 · Cited by 8,285
Guidance for conducting systematic scoping reviews
Peters, M. D. J., Godfrey, C. M., Khalil, H., McInerney, P., Parker, D., & Soares, C. B. (2015) · International Journal of Evidence-Based Healthcare · doi:10.1097/xeb.0000000000000050 · Cited by 7,472
Munn, Z., Moola, S., Lisy, K., Riitano, D., & Tufanaru, C. (2015) · International Journal of Evidence-Based Healthcare · doi:10.1097/xeb.0000000000000054 · Cited by 3,131
How should meta‐regression analyses be undertaken and interpreted?
Thompson, S. G., & Higgins, J. P. T. (2002) · Statistics in Medicine · doi:10.1002/sim.1187 · Cited by 3,026
GRADE guidelines: 4. Rating the quality of evidence—study limitations (risk of bias)
Guyatt, G. H., Oxman, A. D., Vist, G., Kunz, R., Brozek, J., Alonso-Coello, P., Montori, V., Akl, E. A., Djulbegovic, B., & Falck-Ytter, Y. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2010.07.017 · Cited by 2,798
Research works printed but not linked: 2
- The Cochrane Library. http://www.cochranelibrary.com . Accessed 20 Jun 2017.
- Naumann. How to develop a search strategy. 2007. https://www.researchgate.net/file.PostFileLoader.html?id=5996c4eb48954ceeca4bf303&assetKey=AS%3A528656902483968%401503053035454 . Accessed 13 Feb 2018.
Cited by
Brauer, M., Roth, G. A., Aravkin, A. Y., Zheng, P., Abate, K. H., Abate, Y. H., Abbafati, C., Abbasgholizadeh, R., Abbasi, M. A., Abbasian, M., et al. (2024) · The Lancet · doi:10.1016/s0140-6736(24)00933-4 · Cited by 2,884
MOOSE Reporting Guidelines for Meta-analyses of Observational Studies
Brooke, B. S., Schwartz, T. A., & Pawlik, T. M. (2021) · JAMA Surgery · doi:10.1001/jamasurg.2021.0522 · Cited by 742
Dekkers, O. M., Vandenbroucke, J. P., Cevallos, M., Renehan, A. G., Altman, D. G., & Egger, M. (2019) · PLOS Medicine · doi:10.1371/journal.pmed.1002742 · Cited by 525
Self-citationQuantitative Research: A Successful Investigation in Natural and Social Sciences
Mohajan, H. K. (2020) · Journal of Economic Development, Environment and People · doi:10.26458/jedep.v9i4.679 · Cited by 469
Sociodemographic determinants of digital health literacy: A systematic review and meta-analysis
Estrela, M., Semedo, G., Roque, F., Ferreira, P. L., & Herdeiro, M. T. (2023) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2023.105124 · Cited by 414
Metadata
- Authors
- Mueller, M., D’Addario, M., Egger, M., Cevallos, M., Dekkers, O., Mugglin, C., & Scott, P.
- Affiliations
- University of Bern (CH)
- Leiden University Medical Center (NL)
- University of Otago (NZ)
- Corresponding author
- Scott, Pippa
- Venue
- BMC Medical Research Methodology
- Published
- 2018-05-21
- Volume
- 18
- Issue
- 1
- Pages
- 44-44
- ISSN
- 1471-2288 (ISSN-L)
- PMC
- PMC5963098
- OpenAlex
- W2806453502
- Semantic Scholar
- 639ae139189516144a153fcc4b5d862e76db4eae
- Cited by
- 400
- References
- 104
- Citations by year
- 2026: 28 · 2025: 28 · 2024: 55 · 2023: 54 · 2022: 89 · 2021: 71 · 2020: 58 · 2019: 11 · 2018: 4
- FWCI
- 7.08
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- creativecommons.org/licenses/by/4.0/ (tdm)
- Locations
- BMC Medical Research Methodology (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, other-oa)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Leiden Repository (Leiden University) (submittedVersion, public-domain)
- Data Archiving and Networked Services (DANS) (submittedVersion, other-oa)
- BORIS (University Library Bern) (submittedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Leiden Repository (Leiden University) (submittedVersion)
- Open Access CRIS of the University of Bern
- Funders
- University of Otago
- University of Bern
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Physical Sciences › Mathematics › Statistics and Probability › Advanced Causal Inference Techniques
- Keywords
- meta-analysis, observational studies, methodological recommendations, risk of bias assessment, publication bias, heterogeneity, study design heterogeneity, data extraction, search strategies, protocol design, evidence synthesis
- MeSH terms
- Systematic Reviews as Topic, Humans, Publications, Meta-Analysis as Topic, Guidelines as Topic, Observational Studies as Topic
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-04, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-04 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Methods to systematically review and meta-analyse observational studies: a systematic scoping review of recommendations. https://app.cochat.ai/papers/10.1186/s12874-018-0495-9
- The paper
- Mueller, M., D’Addario, M., Egger, M., Cevallos, M., Dekkers, O., Mugglin, C., & Scott, P. (2018). Methods to systematically review and meta-analyse observational studies: a systematic scoping review of recommendations. BMC Medical Research Methodology. https://doi.org/10.1186/s12874-018-0495-9
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.
