Evidence-Based Medicine, Systematic Reviews, and Guidelines in Interventional Pain Management: Part 6. Systematic Reviews and Meta-Analyses of Observational Studies
Abstract
Observational studies provide an important source of information when randomized controlled trials (RCTs) cannot or should not be undertaken, provided that the data are analyzed and interpreted with special attention to bias. Evidence-based medicine (EBM) stresses the examination of evidence from clinical research and describes it as a shift in medical paradigm, in contrast to intuition, unsystematic clinical experience, and pathophysiologic rationale. While the importance of randomized trials has been created by the concept of the hierarchy of evidence in guiding therapy, much of the medical research is observational. The reporting of observational research is often not detailed and clear enough with insufficient quality and poor reporting, which hampers the assessment of strengths and weaknesses of the study and the generalizability of the mixed results. Thus, in recent years, progress and innovations in health care are measured by systematic reviews and meta-analyses. A systematic review is defined as, “the application of scientific strategies that limit bias by the systematic assembly, clinical appraisal, and synthesis of all relevant studies on a specific topic.” Meta-analysis usually is the final step in a systematic review. Systematic reviews and meta-analyses are labor intensive, requiring expertise in both the subject matter and review methodology, and also must follow the rules of EBM which suggests that a formal set of rules must complement medical training and common sense for clinicians to integrate the results of clinical research effectively. While expertise in the review methods is important, the expertise in the subject matter and technical components is also crucial. Even though, systematic reviews and meta-analyses, specifically of RCTs, have exploded, the quality of the systematic reviews is highly variable and consequently, the opinions reached of the same studies are quite divergent. Numerous deficiencies have been described in methodologic assessment of the quality of the individual articles. Consequently, observational studies can provide an important complementary source of information, provided that the data are analyzed and interpreted in the context of confounding bias to which they are prone. Appropriate systematic reviews of observational studies, in conjunction with RCTs, may provide the basis for elimination of a dangerous discrepancy between the experts and the evidence. Steps in conducting systematic reviews of observational studies include planning, conducting, reporting, and disseminating the results. MOOSE, or Meta-analysis of Observational Studies in Epidemiology, a proposal for reporting contains specifications including background, search strategy, methods, results, discussion, and conclusion. Use of the MOOSE checklist should improve the usefulness of meta-analysis for authors, reviewers, editors, readers, and decision-makers. This manuscript describes systematic reviews and meta-analyses of observational studies. Authors frequently utilize RCTs and observational studies in one systematic review; thus, they should also follow the reporting standards of the Quality of Reporting of Meta-analysis (QUOROM) statement, which also provides a checklist. A combined approach of QUOROM and MOOSE will improve reporting of systematic reviews and lead to progress and innovations in health care. Key words: Observational studies, evidence-based medicine, systematic reviews, metaanalysis, randomized trials, case-control studies, cross-sectional studies, cohort studies, confounding bias, QUOROM, MOOSE
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Two articles recommended that the review question state which study designs will be considered in the review.
“Two articles recommended that the review question state which study designs will be considered in the review [47, 48].”
From Methods to systematically review and meta-analyse observational studies: a systematic scoping review of recommendations (2018) · cites this paper as [47]
There was some disagreement among the articles whether the research question should be specific (narrowly formulated) or general (broadly formulated).
“There was some disagreement (Table 4) whether the research question should be specific (narrowly formulated) [51, 53], or general (broadly formulated) [35, 47, 49].”
From Methods to systematically review and meta-analyse observational studies: a systematic scoping review of recommendations (2018) · cites this paper as [47]
Aspects frequently mentioned as important to address in the research question were the population, exposures or interventions, and outcomes.
“Aspects that were frequently mentioned as important to address were the population, exposures or interventions, and outcomes [38–41, 43, 47, 48, 50, 54].”
From Methods to systematically review and meta-analyse observational studies: a systematic scoping review of recommendations (2018) · cites this paper as [47]
Many articles recommended that the eligibility criteria need to be pre-specified and that the rationale for defining the eligibility criteria should be explicitly justified, unambiguous, and derived from the review question.
“Many recommended that the eligibility criteria need to be pre-specified [30, 33, 38, 39, 47, 55, 65] and that the rationale for defining the eligibility criteria should be explicitly justified [38, 39, 65], unambiguous [65], and derived from the review question [47].”
From Methods to systematically review and meta-analyse observational studies: a systematic scoping review of recommendations (2018) · cites this paper as [47]
It was suggested that eligibility criteria should be defined in terms of population, interventions, outcomes and study design of interest, but a modified version for reviews of observational studies was also proposed: condition, context and population ("CoCoPop").
“Similar to reviews of RCTs, it was suggested that criteria should be defined in terms of population, interventions, outcomes and study design of interest [47, 54], but a modified version for reviews of observational studies was also proposed: condition, context and population (“CoCoPop”) [72].”
From Methods to systematically review and meta-analyse observational studies: a systematic scoping review of recommendations (2018) · cites this paper as [47]
About one third of the articles acknowledged that searching for observational studies requires additional time and resources because of lack of specific search filters and poorly established or inconsistently used indexing terms.
“About one third of the articles acknowledged that searching for observational studies requires additional time and resources because of lack of specific search filters and poorly established or inconsistently used indexing terms [38, 39, 47, 49, 56, 58, 59, 64, 65].”
From Methods to systematically review and meta-analyse observational studies: a systematic scoping review of recommendations (2018) · cites this paper as [47]
Nine articles (10%) made recommendations on how to perform data extraction, with publication dates between 1995 and 2016 (median year 2009).
“Nine articles (10%) made recommendations on how to perform data extraction [29, 33–35, 38, 43, 47, 48, 74], with publication dates between 1995 and 2016 (median year of publication 2009).”
From Methods to systematically review and meta-analyse observational studies: a systematic scoping review of recommendations (2018) · cites this paper as [47]
The need for complete reporting of all relevant methodological aspects as a prerequisite to assessing the risk of bias of studies was highlighted in 10 articles, and was the focus of recommendations in a publication from the MOOSE group.
“The need for complete reporting of all relevant methodological aspects as a prerequisite to assessing the risk of bias of studies was highlighted in 10 articles, [16, 33, 35, 47, 50, 56, 86, 87, 90, 91] and was the focus of recommendations in a publication from the Meta-analysis Of Observational Studies in Epidemiology (MOOSE) group [12].”
From Methods to systematically review and meta-analyse observational studies: a systematic scoping review of recommendations (2018) · cites this paper as [47]
Many considered heterogeneity important for the decision whether to pool the data.
“Many considered heterogeneity important for the decision whether to pool the data [34, 36, 47, 55, 56, 66, 101, 103].”
From Methods to systematically review and meta-analyse observational studies: a systematic scoping review of recommendations (2018) · cites this paper as [47]
Authors recommended not to pool if heterogeneity is too high, and use statistical measures of heterogeneity to choose between random-effects and fixed-effect models.
“Authors recommended not to pool if heterogeneity is too high [35, 38, 47, 85, 103], and use statistical measures of heterogeneity to choose between random-effects (in case of heterogeneity) and fixed-effect models [55, 66].”
From Methods to systematically review and meta-analyse observational studies: a systematic scoping review of recommendations (2018) · cites this paper as [47]
The citation lists haven’t been fetched yet
References 293 · Cited by 168
- Authors
- Manchikanti, L.
- Affiliations
- University of Louisville (US)
- Corresponding author
- Manchikanti, Laxmaiah
- Venue
- Pain Physician
- Published
- 2008-09-15
- Volume
- 5;12
- Issue
- 5;9
- Pages
- 819-850
- ISSN
- 1533-3159 (ISSN-L)
- 2150-1149
- OpenAlex
- W2336685384
- Cited by
- 168
- References
- 293
- Citations by year
- 2026: 1 · 2025: 1 · 2024: 19 · 2023: 5 · 2022: 7 · 2021: 7 · 2020: 3 · 2019: 6 · 2018: 9 · 2017: 7 · 2016: 6 · 2015: 5 · 2014: 11 · 2013: 18 · 2012: 18
- FWCI
- 17.02
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Pain Physician (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Clinical practice guidelines implementation
- Health Sciences › Health Professions › General Health Professions › Health Sciences Research and Education
- Keywords
- interventional pain management, systematic reviews, observational studies, confounding, MOOSE checklist, reporting quality, evidence-based medicine, evidence hierarchy, randomized controlled trials, bias assessment, reporting bias, publication bias, study quality assessment, generalizability of results, knowledge-practice gap
- MeSH terms
- Clinical Protocols, Data Interpretation, Statistical, Humans, Review Literature as Topic, Meta-Analysis as Topic, Reproducibility of Results, Observer Variation, Randomized Controlled Trials as Topic, Outcome Assessment, Health Care, Practice Guidelines as Topic, Evidence-Based Medicine, Pain Management
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
