The DOI matches its Crossref record; 25 of the 26 references it cites were found in a registry or an index.
Systematic reviews of adverse effects: framework for a structured approach
Open accessAbstract
Background: As every healthcare intervention carries some risk of harm, clinical decision making needs to be supported by a systematic assessment of the balance of benefit to harm. A systematic review that considers only the favourable outcomes of an intervention, without also assessing the adverse effects, can mislead by introducing a bias favouring the intervention. Much of the current guidance on systematic reviews is directed towards the evaluation of effectiveness; but this differs in important ways from the methods used in assessing the safety and tolerability of an intervention. A detailed discussion of why, how and when to include adverse effects in a systematic review, is required. Methods: This discussion paper, which presupposes a basic knowledge of systematic review methodology, was developed by consensus among experienced reviewers, members of the Adverse Effects Subgroup of The Cochrane Collaboration, and supplemented by a consultation of content experts in reviews methodology, as well as those working in drug safety. Results: A logical framework for making decisions in reviews that incorporate adverse effects is provided. We explore situations where a comprehensive investigation of adverse effects is warranted and suggest strategies to identify practicable and clinically useful outcomes. The advantages and disadvantages of including observational and experimental study designs are reviewed. The consequences of including separate studies for intended and unintended effects are explained. Detailed advice is given on designing electronic searches for studies with adverse effects data. Reviewers of adverse effects are given general guidance on the assessment of study bias, data collection, analysis, presentation and the interpretation of harms in a systematic review. Conclusion: Readers need to be able to recognize how strategic choices made in the review process determine what harms are found, and how the findings may affect clinical decisions. Researchers undertaking a systematic review that incorporates adverse effect data should understand the rationale for the suggested methods and be able to implement them in their review.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Nine articles (10%) made recommendations on how to perform data extraction [29, 33–35, 38, 43, 47, 48, 74], with publication dates between 1995 and 2016 (median year of publication 2009).”
From Methods to systematically review and meta-analyse observational studies: a systematic scoping review of recommendations (2018) · cites this paper as [35]
The cited source says what the claim attributes to it.
In the citing paper
“As this is the first systematic review of the literature involving meditation adverse events (MAEs), we utilized the 'broad sweep' approach recommended by the Cochrane Adverse Effects Methods Group (27) in order to gain a general view of the variety of adverse events, as well as attempting a preliminary estimate of their prevalence.”
From Adverse events in meditation practices and meditation‐based therapies: a systematic review (2020) · cites this paper as [27]
In the citing paper
“Twenty articles (22%) gave recommendations on the key item “research question” [29, 30, 35, 36, 38–43, 45–54], with publication dates between 1997 and 2015 (median year of publication 2009).”
From Methods to systematically review and meta-analyse observational studies: a systematic scoping review of recommendations (2018) · cites this paper as [35]
In the citing paper
“Sixteen articles (17%) provided recommendations on the key item “protocol development” (Table 3), [29–44] with publication dates between 1994 and 2015 (median year of publication 2009).”
From Methods to systematically review and meta-analyse observational studies: a systematic scoping review of recommendations (2018) · cites this paper as [35]
In the citing paper
“There was some disagreement (Table 4) whether the research question should be specific (narrowly formulated) [51, 53], or general (broadly formulated) [35, 47, 49].”
From Methods to systematically review and meta-analyse observational studies: a systematic scoping review of recommendations (2018) · cites this paper as [35]
In the citing paper
“The articles recommended that the protocol should state the objectives, hypotheses to be tested and the rationale of the review, [29] and that it should describe eligibility criteria [29, 33, 35, 36, 38, 39, 41], define the type of studies to be included [35, 37, 42, 44], and give the reason when including observational studies in the review [35, 37, 40].”
From Methods to systematically review and meta-analyse observational studies: a systematic scoping review of recommendations (2018) · cites this paper as [35]
In the citing paper
“Thirty-three articles (35%) made recommendations about the key item “search strategy” [12, 29, 30, 33–36, 38, 39, 43, 46–51, 53–69], with publication dates between 1995 and 2016 (median year of publication 2007).”
From Methods to systematically review and meta-analyse observational studies: a systematic scoping review of recommendations (2018) · cites this paper as [35]
In the citing paper
“The majority of articles discussed aspects general to systematic reviews including advantages and limitations of different literature sources and databases, search tools, the importance of identifying unpublished studies, literature searching techniques including how to build a search string and reporting [12, 29, 33–36, 46, 48, 50, 51, 53, 54, 57, 59–62, 67, 68].”
From Methods to systematically review and meta-analyse observational studies: a systematic scoping review of recommendations (2018) · cites this paper as [35]
In the citing paper
“Twenty-five (27%) articles gave recommendations on item “considering different study designs” [8, 29, 34, 35, 37–39, 42, 44, 49, 51, 56, 61, 65, 70, 73, 75, 76, 77–83], with publication dates between 1995 and 2016 (median year of publication 2007).”
From Methods to systematically review and meta-analyse observational studies: a systematic scoping review of recommendations (2018) · cites this paper as [35]
In the citing paper
“Five articles discouraged combining different study designs in meta-analyses but saw the added value of including different designs in a systematic review [35, 38, 56, 61, 75].”
From Methods to systematically review and meta-analyse observational studies: a systematic scoping review of recommendations (2018) · cites this paper as [35]
In the citing paper
“Most articles considered the combination of RCTs and non-randomised designs [29, 35, 38, 39, 49, 51, 56, 61, 75, 81].”
From Methods to systematically review and meta-analyse observational studies: a systematic scoping review of recommendations (2018) · cites this paper as [35]
In the citing paper
“Authors agreed that combining different study designs in a meta-analysis can lead to misleading results if done without careful consideration [35, 38, 56, 70, 75, 76].”
From Methods to systematically review and meta-analyse observational studies: a systematic scoping review of recommendations (2018) · cites this paper as [35]
In the citing paper
“The need for complete reporting of all relevant methodological aspects as a prerequisite to assessing the risk of bias of studies was highlighted in 10 articles, [16, 33, 35, 47, 50, 56, 86, 87, 90, 91] and was the focus of recommendations in a publication from the Meta-analysis Of Observational Studies in Epidemiology (MOOSE) group [12].”
From Methods to systematically review and meta-analyse observational studies: a systematic scoping review of recommendations (2018) · cites this paper as [35]
In the citing paper
“Thirty-nine articles (42%) made recommendations on the assessment of quality and risk of bias of individual studies, [12, 16, 29, 30, 32–35, 37, 38, 43, 47–50, 52–54, 56, 64–66, 70, 71, 73, 75, 77, 82, 84–94] with publication dates between 1994 and 2016 (median year of publication 2009).”
From Methods to systematically review and meta-analyse observational studies: a systematic scoping review of recommendations (2018) · cites this paper as [35]
In the citing paper
“Fifty-two articles (56%), provided recommendations on the statistical analysis in the context of meta-analysis of observational studies [8, 12, 16, 29–31, 33–36, 38, 41–44, 47, 48, 52, 54–56, 61, 66, 70, 72, 73, 77, 79–83, 85, 87, 96, 100, 101, 103–105, 107–118], with publication dates between 1994 and 2016 (median year of publication 2006).”
From Methods to systematically review and meta-analyse observational studies: a systematic scoping review of recommendations (2018) · cites this paper as [35]
In the citing paper
“Authors recommended not to pool if heterogeneity is too high [35, 38, 47, 85, 103], and use statistical measures of heterogeneity to choose between random-effects (in case of heterogeneity) and fixed-effect models [55, 66].”
From Methods to systematically review and meta-analyse observational studies: a systematic scoping review of recommendations (2018) · cites this paper as [35]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
(2014) · Medical Entomology and Zoology · Cited by 17,633
Assessing the quality of controlled clinical trials
Jüni, P., Altman, D. G., & Egger, M. (2001) · BMJ · doi:10.1136/bmj.323.7303.42 · Cited by 3,309
Evaluating non-randomised intervention studies
Deeks, J., Dinnes, J., D'Amico, R., Sowden, A., Sakarovitch, C., Song, F., Petticrew, M., & Altman, D. (2003) · Health Technology Assessment · doi:10.3310/hta7270 · Cited by 2,907
Charbonneau, D. H. (2005) · Europe PMC (PubMed Central) · Cited by 2,041
Better Reporting of Harms in Randomized Trials: An Extension of the CONSORT Statement
Ioannidis, J. P. A., Evans, S. J. W., Gøtzsche, P. C., O'Neill, R. T., Altman, D. G., Schulz, K., Moher, D., & for the CONSORT Group* (2004) · Annals of Internal Medicine · doi:10.7326/0003-4819-141-10-200411160-00009 · Cited by 1,593
Risk of gastrointestinal haemorrhage with long term use of aspirin: meta-analysis
Derry, S. (2000) · BMJ · doi:10.1136/bmj.321.7270.1183 · Cited by 675
Probability of adverse events that have not yet occurred: a statistical reminder
Eypasch, E. P., Lefering, R., Kum, C. K., & Troidl, H. (1995) · BMJ · doi:10.1136/bmj.311.7005.619 · Cited by 342
Comparison of evidence on harms of medical interventions in randomized and nonrandomized studies
Papanikolaou, P. N. (2006) · Canadian Medical Association Journal · doi:10.1503/cmaj.050873 · Cited by 229
Bent, S., Padula, A., & Avins, A. L. (2006) · Annals of Internal Medicine · doi:10.7326/0003-4819-144-4-200602210-00007 · Cited by 177
Reporting of Adverse Effects in Clinical Trials Should Be Improved
Edwards, J. E., McQuay, H. J., Moore, R. A., & Collins, S. (1999) · Journal of Pain and Symptom Management · doi:10.1016/s0885-3924(99)00093-7 · Cited by 165
Reporting of adverse drug reactions in randomised controlled trials – a systematic survey
Loke, Y. K., & Derry, S. (2001) · BMC Clinical Pharmacology · doi:10.1186/1472-6904-1-3 · Cited by 128
Self-citationDeveloping efficient search strategies to identify reports of adverse effects in medline and embase
Golder, S., McIntosh, H. M., Duffy, S., & Glanville, J. (2006) · Health Information & Libraries Journal · doi:10.1111/j.1471-1842.2006.00634.x · Cited by 117
Derry, S., Loke, Y. K., & Aronson, J. K. (2001) · BMC Medical Research Methodology · doi:10.1186/1471-2288-1-7 · Cited by 110
Self-citationCase reports of suspected adverse drug reactions—systematic literature survey of follow-up
Loke, Y. K., Price, D., Derry, S., & Aronson, J. K. (2006) · BMJ · doi:10.1136/bmj.38701.399942.63 · Cited by 98
Self-citationUnintended events following immunization with MMR: a systematic review
Jefferson, T., Price, D., Demicheli, V., & Bianco, E. (2003) · Vaccine · doi:10.1016/s0264-410x(03)00271-8 · Cited by 86
Self-citationMedawar, C., & Herxheimer, A. (2004) · International Journal of Risk & Safety in Medicine · Cited by 80
Self-citationOLSEN, T. K. H. (1999) · Blood Pressure · doi:10.1080/080370599438266 · Cited by 74
Room for improvement? A survey of the methods used in systematic reviews of adverse effects
Golder, S., Loke, Y. K., & McIntosh, H. M. (2006) · BMC Medical Research Methodology · doi:10.1186/1471-2288-6-3 · Cited by 73
Self-citationAssessing harmful effects in systematic Reviews
McIntosh, H. M., Woolacott, N. F., & Bagnall, A. (2004) · BMC Medical Research Methodology · doi:10.1186/1471-2288-4-19 · Cited by 66
Loke, Y. K., Derry, S., & Aronson, J. K. (2004) · British Journal of Clinical Pharmacology · doi:10.1111/j.0306-5251.2003.02055.x · Cited by 60
Self-citationSingh, S., & Loke, Y. K. (2006) · Drug Safety · doi:10.2165/00002018-200629120-00004 · Cited by 50
Self-citationTeleoanalysis: combining data from different types of study
Wald, N. J., & Morris, J. K. (2003) · BMJ · doi:10.1136/bmj.327.7415.616 · Cited by 49
Assessing the benefit-harm balance at the bedside
Loke, Y. K. (2004) · BMJ · doi:10.1136/bmj.329.7456.7 · Cited by 17
Self-citationCompleteness of Safety Reporting in Randomized Trials: An Evaluation of 7 Medical Areas
Ioannidis, J. P. A., & Lau, J. (2001) · JAMA · doi:10.1001/jama.285.4.437 · Cited by 3
Balancing benefits and harms in health care:
Jefferson, T., & Demicheli, V. (2003) · BMJ · doi:10.1136/bmj.327.7417.750-b · Cited by 3
Cited by
Sorted by citations
Systematic reviews: CRD's guidance for undertaking reviews in health care
Tacconelli, E. (2010) · The Lancet Infectious Diseases · doi:10.1016/s1473-3099(10)70065-7 · Cited by 3,270
Thangaratinam, S., Rogozinska, E., Jolly, K., Glinkowski, S., Roseboom, T., Tomlinson, J. W., Kunz, R., Mol, B. W., Coomarasamy, A., & Khan, K. S. (2012) · BMJ · doi:10.1136/bmj.e2088 · Cited by 802
Side-effects associated with ketamine use in depression: a systematic review
Short, B., Fong, J., Galvez, V., Shelker, W., & Loo, C. K. (2018) · The Lancet Psychiatry · doi:10.1016/s2215-0366(17)30272-9 · Cited by 558
PRISMA harms checklist: improving harms reporting in systematic reviews
Zorzela, L., Loke, Y. K., Ioannidis, J. P., Golder, S., Santaguida, P., Altman, D. G., Moher, D., Vohra, S., & PRISMA harms group (2016) · BMJ · doi:10.1136/bmj.i157 · Cited by 557
Self-citationWater fluoridation for the prevention of dental caries
Iheozor-Ejiofor, Z., Worthington, H. V., Walsh, T., O'Malley, L., Clarkson, J. E., Macey, R., Alam, R., Tugwell, P., Welch, V., & Glenny, A.-M. (2015) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd010856.pub2 · Cited by 214
Metadata
- Authors
- Loke, Y. K., Price, D., Herxheimer, A., & the Cochrane Adverse Effects Methods Group
- Affiliations
- University of East Anglia (GB)
- University of Oxford (GB)
- Cochrane (GB)
- Corresponding author
- Herxheimer, Andrew
- ORCID
- Loke, Yoon K
- Venue
- BMC Medical Research Methodology
- Published
- 2007-07-05
- Volume
- 7
- Issue
- 1
- Pages
- 32-32
- ISSN
- 1471-2288 (ISSN-L)
- PMC
- PMC1933432
- OpenAlex
- W2052762949
- Cited by
- 253
- References
- 26
- Citations by year
- 2026: 8 · 2025: 12 · 2024: 9 · 2023: 8 · 2022: 9 · 2021: 9 · 2020: 15 · 2019: 19 · 2018: 11 · 2017: 19 · 2016: 20 · 2015: 19 · 2014: 15 · 2013: 19 · 2012: 11
- FWCI
- 1.74
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- BMC Medical Research Methodology (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
- UEA Digital Repository (University of East Anglia) (submittedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Life Sciences › Pharmacology, Toxicology and Pharmaceutics › Toxicology › Pharmacovigilance and Adverse Drug Reactions
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Keywords
- adverse effects, harm assessment, systematic review methodology, drug safety, benefit-harm balance
- MeSH terms
- Drug Evaluation, Humans, Research Design, Meta-Analysis as Topic, Randomized Controlled Trials as Topic, Databases, Bibliographic, Information Storage and Retrieval, Practice Guidelines as Topic, Risk Assessment, Decision Support Systems, Clinical, Consensus, Medical Subject Headings, Drug-Related Side Effects and Adverse Reactions
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review