AMSTAR is a reliable and valid measurement tool to assess the methodological quality of systematic reviews
Open accessAbstract
Objective Our purpose was to measure the agreement, reliability, construct validity, and feasibility of a measurement tool to assess systematic reviews (AMSTAR).Study design and setting We randomly selected 30 systematic reviews from a database. Each was assessed by two reviewers using: (1) the enhanced quality assessment questionnaire (Overview of Quality Assessment Questionnaire [OQAQ]); (2) Sacks' instrument; and (3) our newly developed measurement tool (AMSTAR). We report on reliability (interobserver kappas of the 11 AMSTAR items), intraclass correlation coefficients (ICCs) of the sum scores, construct validity (ICCs of the sum scores of AMSTAR compared with those of other instruments), and completion times.Results The interrater agreement of the individual items of AMSTAR was substantial with a mean kappa of 0.70 (95% confidence interval [CI]: 0.57, 0.83) (range: 0.38-1.0). Kappas recorded for the other instruments were 0.63 (95% CI: 0.38, 0.78) for enhanced OQAQ and 0.40 (95% CI: 0.29, 0.50) for the Sacks' instrument. The ICC of the total score for AMSTAR was 0.84 (95% CI: 0.65, 0.92) compared with 0.91 (95% CI: 0.82, 0.96) for OQAQ and 0.86 (95% CI: 0.71, 0.94) for the Sacks' instrument. AMSTAR proved easy to apply, each review taking about 15 minutes to complete.Conclusions AMSTAR has good agreement, reliability, construct validity, and feasibility. These findings need confirmation by a broader range of assessors and a more diverse range of reviews.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Two independent authors assessed the quality of included systematic reviews and meta-analyses using the AMSTAR and AMSTAR+ tools.
“two independent authors assessed the quality of the systematic reviews and meta-analyses using the AMSTAR and AMSTAR+ tool [53][54][55].”
From Exercise as Medicine for Mental and Substance Use Disorders: A Meta-review of the Benefits for Neuropsychiatric and Cognitive Outcomes (2020) · cites this paper as [53]
The methodological quality of the reviews was examined using the AMSTAR tool.
“The methodological quality of these reviews was examined using the AMSTAR tool [38].”
From The effectiveness of mindfulness-based interventions on anxiety disorders. A systematic meta-review (2020) · cites this paper as [38]
The validity and reliability of the AMSTAR tool has been established in previous studies.
“The validity and reliability of the AMSTAR has been established in previous studies.19,20”
From Quality assessment of systematic reviews on the efficacy of oral appliance therapy for adult and pediatric sleep-disordered breathing (2016) · cites this paper as [19]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
STATISTICAL METHODS FOR ASSESSING AGREEMENT BETWEEN TWO METHODS OF CLINICAL MEASUREMENT
Martin Bland, J., & Altman, D. (1986) · The Lancet · doi:10.1016/s0140-6736(86)90837-8 · Cited by 47,965 · see citing works
A Coefficient of Agreement for Nominal Scales
Cohen, J. (1960) · Educational and Psychological Measurement · doi:10.1177/001316446002000104 · Cited by 42,398 · see citing works
Weighted kappa: nominal scale agreement provision for scaled disagreement or partial credit
Cohen, J. (1968) · Psychological Bulletin · doi:10.1037/h0026256 · Cited by 8,736 · see citing works
Moher, D., Cook, D. J., Eastwood, S., Olkin, I., Rennie, D., & Stroup, D. F. (1999) · The Lancet · doi:10.1016/s0140-6736(99)04149-5 · Cited by 4,826 · see citing works
Development of AMSTAR: A measurement tool to assess the methodological quality of systematic reviews
Shea, B. J., Grimshaw, J. M., Wells, G. A., Boers, M., Andersson, N., Hamel, C., Porter, A. C., Tugwell, P., Moher, D., & Bouter, L. M. (2007) · BMC Medical Research Methodology · doi:10.1186/1471-2288-7-10 · Cited by 4,374 · see citing works
Self-citationAssessing the quality of controlled clinical trials
Jüni, P., Altman, D. G., & Egger, M. (2001) · BMJ · doi:10.1136/bmj.323.7303.42 · Cited by 3,309 · see citing works
Users' guides to the medical literature
Guyatt, G. H. (1993) · JAMA · doi:10.1001/jama.270.17.2096 · Cited by 2,970 · see citing works
Selective publication of antidepressant trials and its influence on apparent efficacy
Turner, E. H., Matthews, A. M., Linardatos, E., Tell, R. A., & Rosenthal, R. (2008) · New England Journal of Medicine · doi:10.1056/nejmsa065779 · Cited by 2,435 · see citing works
Systematic Reviews in Health Care : Meta-Analysis in Context
Egger, M., Smith, G. D., & Altman, D. G. (2000) · Cited by 2,312 · see citing works
Systematic Reviews in Health Care. Meta-analysis in Context.
Kleijnen, J., & Antes, G. (2002) · International Journal of Epidemiology · doi:10.1093/ije/31.3.697 · Cited by 1,373 · see citing works
Moher, D., Jadad, A. R., Nichol, G., Penman, M., Tugwell, P., & Walsh, S. (1995) · Controlled Clinical Trials · doi:10.1016/0197-2456(94)00031-w · Cited by 1,308 · see citing works
Meta-Analyses of Randomized Controlled Trials
Sacks, H. S., Berrier, J., Reitman, D., Ancona-Berk, V. A., & Chalmers, T. C. (1987) · New England Journal of Medicine · doi:10.1056/nejm198702193160806 · Cited by 1,253 · see citing works
Whittington, C. J., Kendall, T., Fonagy, P., Cottrell, D. J., Cotgrove, A. J., & Boddington, E. (2004) · The Lancet · doi:10.1016/s0140-6736(04)16043-1 · Cited by 1,055 · see citing works
Effects of Adjuvant Tamoxifen and of Cytotoxic Therapy on Mortality in Early Breast Cancer
Early Breast Cancer Trialists' Collaborative Group (1988) · New England Journal of Medicine · doi:10.1056/nejm198812293192601 · Cited by 934 · see citing works
Validation of an index of the quality of review articles
Oxman, A. D., & Guyatt, G. H. (1991) · Journal of Clinical Epidemiology · doi:10.1016/0895-4356(91)90160-b · Cited by 788 · see citing works
Prevention of Venous Thromboembolism in General Surgical Patients
Clagett, G. P., & REISCH, J. S. (1988) · Annals of Surgery · doi:10.1097/00000658-198808000-00016 · Cited by 688 · see citing works
External Validation of a Measurement Tool to Assess Systematic Reviews (AMSTAR)
Shea, B. J., Bouter, L. M., Peterson, J., Boers, M., Andersson, N., Ortiz, Z., Ramsay, T., Bai, A., Shukla, V. K., & Grimshaw, J. M. (2007) · PLoS ONE · doi:10.1371/journal.pone.0001350 · Cited by 569 · see citing works
Self-citationBalk, E. M. (2002) · JAMA · doi:10.1001/jama.287.22.2973 · Cited by 478 · see citing works
Why review articles on the health effects of passive smoking reach different conclusions
Barnes, D. E. (1998) · JAMA · doi:10.1001/jama.279.19.1566 · Cited by 442 · see citing works
Evidence Favoring the Use of Anticoagulants in the Hospital Phase of Acute Myocardial Infarction
Chalmers, T. C., Matta, R. J., Smith, H., & Kunzler, A.-M. (1977) · New England Journal of Medicine · doi:10.1056/nejm197711172972004 · Cited by 423 · see citing works
Hughes, E. G., Fedorkow, D. M., Daya, S., Sagle, M. A., de Koppel, P. V., & Collins, J. A. (1992) · Fertility and Sterility · doi:10.1016/s0015-0282(16)55430-2 · Cited by 401 · see citing works
Appel, L. J. (1993) · Archives of Internal Medicine · doi:10.1001/archinte.153.12.1429 · Cited by 380 · see citing works
Crystalloid versus colloid fluid resuscitation: a meta-analysis of mortality.
Velanovich, V. (1989) · PubMed · doi:10.5555/uri:pii:003960608990010x · Cited by 254 · see citing works
Diversity of decision-making models and the measurement of interrater agreement
Uebersax, J. S. (1987) · Psychological Bulletin · doi:10.1037/0033-2909.101.1.140 · Cited by 254 · see citing works
Pérez-Escamilla, R., Pollitt, E., Lönnerdal, B. L., & Dewey, K. G. (1994) · American Journal of Public Health · doi:10.2105/ajph.84.1.89 · Cited by 221 · see citing works
Magnesium sulphate and other anticonvulsants for women with pre-eclampsia
Duley, L., Gülmezoglu, A. M., & Henderson-Smart, D. J. (2003) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd000025 · Cited by 211 · see citing works
Biondi-Zoccai, G. G. L., Lotrionte, M., Abbate, A., Testa, L., Remigi, E., Burzotta, F., Valgimigli, M., Romagnoli, E., Crea, F., & Agostoni, P. (2006) · BMJ · doi:10.1136/bmj.38693.516782.7c · Cited by 149 · see citing works
Challenges in Systematic Reviews That Assess Treatment Harms
Chou, R., & Helfand, M. (2005) · Annals of Internal Medicine · doi:10.7326/0003-4819-142-12_part_2-200506211-00009 · Cited by 147 · see citing works
A methodologic framework for developing and selecting endpoints in clinical trials.
Tugwell, P., & Bombardier, C. (1983) · PubMed · Cited by 137 · see citing works
Treatments of preterm premature rupture of the membranes: A meta-analysis
Ohlsson, A. (1989) · American Journal of Obstetrics and Gynecology · doi:10.1016/0002-9378(89)90306-2 · Cited by 132 · see citing works
Rudy, E. B., Lucke, J. F., Whitman, G. R., et al. (2001) · Cited by 121 · see citing works
Oxman, A. D., Schünemann, H. J., & Fretheim, A. (2006) · Health Research Policy and Systems · doi:10.1186/1478-4505-4-20 · Cited by 116 · see citing works
Assessing the Quality of Reports of Systematic Reviews: The QUOROM Statement Compared to Other Tools
Shea, B., Dubé, C., & Moher, D. (2001) · Wiley eBooks · doi:10.1002/9780470693926.ch7 · Cited by 80 · see citing works
Self-citationSomatostatin v placebo in bleeding oesophageal varices: randomised trial and meta-analysis
Gotzsche, P. C., Gjørup, I. E., Bonnén, H., Brahe, N. E. B., Becker, U., & Burcharth, F. (1995) · BMJ · doi:10.1136/bmj.310.6993.1495 · Cited by 76 · see citing works
Antiplatelet or Anticoagulant Therapy after Coronary Artery Bypass Surgery
Henderson, W. G., Goldman, S., Copeland, J. G., Moritz, T. E., & Harker, L. A. (1989) · Annals of Internal Medicine · doi:10.7326/0003-4819-111-9-743 · Cited by 74 · see citing works
Quality of controlled clinical trials. The case of imaging ultrasound in obstetrics: a review
Thacker, S. B. (1985) · BJOG An International Journal of Obstetrics & Gynaecology · doi:10.1111/j.1471-0528.1985.tb01346.x · Cited by 73 · see citing works
Fanning, J., Bennett, T. Z., & Hilgers, R.-D. (1992) · PubMed · Cited by 67 · see citing works
Hopfenmüller, W. (1994) · PubMed · Cited by 52 · see citing works
Hypertension intervention pooling project.
Kaufmann, P. G., HIPP Investigators, Jacob, R. G., Ewart, C. K., Chesney, M. A., Muenz, L. R., Doub, N., & Mercer, W. (1988) · Health Psychology · doi:10.1037/0278-6133.7.suppl.209 · Cited by 41 · see citing works
Relative Efficacy of Vasodilator Therapy in Chronic Congestive Heart Failure
Mulrow, C. D. (1988) · JAMA · doi:10.1001/jama.1988.03720230032026 · Cited by 40 · see citing works
Gent, M., & Roberts, R. S. (1986) · CHEST Journal · doi:10.1378/chest.89.5_supplement.396s · Cited by 40 · see citing works
Early versus delayed initiation of breastfeeding.
Renfrew, M. J., Lang, S. P., & Woolridge, M. W. (2000) · PubMed · doi:10.1002/14651858.cd000043 · Cited by 39 · see citing works
A meta-analysis of the use of amoxycillin-clavulanic acid in surgical prophylaxis
Wilson, A. P. R., Shrimpton, S. B., & Jaderberg, M. (1992) · Journal of Hospital Infection · doi:10.1016/s0195-6701(05)80003-9 · Cited by 28 · see citing works
Randomized Trials of Aminoglycoside Antibiotics: Quantitative Overview
Buring, J. E., Evans, D. A., Mayrent, S. L., et al. (1988) · Clinical Infectious Diseases · doi:10.1093/clinids/10.5.951 · Cited by 28 · see citing works
Postherpetic Neuralgia and Systemic Corticosteroid Therapy
Lycka, B. A. S. (1990) · International Journal of Dermatology · doi:10.1111/j.1365-4362.1990.tb04852.x · Cited by 26 · see citing works
Pavelka, R., & Salzer, H. (2010) · Gynecologic and Obstetric Investigation · doi:10.1159/000299661 · Cited by 8 · see citing works
Mulrow, C. D. (1988) · JAMA · doi:10.1001/jama.259.23.3422 · Cited by 3 · see citing works
Flagged first, then by citations
Research works printed but not linked: 4
- Appel LJ, Miller ER, Seidler AJ, Whelton PK. Does supplementation of diet with ''fish oil'' reduce blood pressure. Arch Intern Med 1993;153:1429e38.
- Hopfenmuller W. Nackweis der therapeutischen Wirksamkeit eines Ginkgo biloba-Spezial extrakes: Meta-Analyse von 11 klinischen Studien bei Patienten mit Hirnleistungsstorungen im Alter. Arzneimittel-Forschung 1994;44:1005e13.
- Bland JM, Altman DG. Statistical methods for assessing agreement between measurements. Biochim Clin 1987;11:399e404.
- Moher D, Cook DJ, Eastwood S, Olkin I, Rennie D. Stroup D for the QUOROM group. Improving the reporting quality of meta-analysis of randomized controlled trials: the QUOROM statement. Lancet 1999;354:1896e900.
Cited by
Sorted by citations
Shea, B. J., Reeves, B. C., Wells, G., Thuku, M., Hamel, C., Moran, J., Moher, D., Tugwell, P., Welch, V., Kristjansson, E., et al. (2017) · BMJ · doi:10.1136/bmj.j4008 · Cited by 10,636 · see citing works
Self-citationQaseem, A., Wilt, T. J., McLean, R. M., Forciea, M. A., & for the Clinical Guidelines Committee of the American College of Physicians* (2017) · Annals of Internal Medicine · doi:10.7326/m16-2367 · Cited by 3,234 · see citing works
Risks of all-cause and suicide mortality in mental disorders: a meta-review
Chesney, E., Goodwin, G. M., & Fazel, S. (2014) · World Psychiatry · doi:10.1002/wps.20128 · Cited by 2,115 · see citing works
Synthesizing information systems knowledge: A typology of literature reviews
Paré, G., Trudel, M.-C., Jaana, M., & Kitsiou, S. (2015) · Information & Management · doi:10.1016/j.im.2014.08.008 · Cited by 1,987 · see citing works
Nurturing care: Promoting early childhood development
Britto, P. R., Lye, S. J., Proulx, K., Yousafzai, A. K., Matthews, S. G., Vaivada, T., Perez-Escamilla, R., Rao, N., Ip, P., Fernald, L. C. H., et al. (2017) · The Lancet · doi:10.1016/s0140-6736(16)31390-3 · Cited by 1,796 · see citing works
Bozdag, G., Mumusoglu, S., Zengin, D., Karabulut, E., & Yildiz, B. O. (2016) · Human Reproduction · doi:10.1093/humrep/dew218 · Cited by 1,526 · see citing works
Rebar, A. L., Stanton, R., Geard, D., Short, C., Duncan, M. J., & Vandelanotte, C. (2015) · Health Psychology Review · doi:10.1080/17437199.2015.1022901 · Cited by 1,323 · see citing works
Methodology in conducting a systematic review of systematic reviews of healthcare interventions
Smith, V., Devane, D., Begley, C. M., & Clarke, M. (2011) · BMC Medical Research Methodology · doi:10.1186/1471-2288-11-15 · Cited by 1,227 · see citing works
Interventions for improving upper limb function after stroke
Pollock, A., Farmer, S. E., Brady, M. C., Langhorne, P., Mead, G. E., Mehrholz, J., & van Wijck, F. (2014) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd010820.pub2 · Cited by 909 · see citing works
Panic, N., Leoncini, E., de Belvis, G., Ricciardi, W., & Boccia, S. (2013) · PLoS ONE · doi:10.1371/journal.pone.0083138 · Cited by 883 · see citing works
Chou, R., Deyo, R., Friedly, J., Skelly, A., Hashimoto, R., Weimer, M., Fu, R., Dana, T., Kraegel, P., Griffin, J., et al. (2017) · Annals of Internal Medicine · doi:10.7326/m16-2459 · Cited by 855 · see citing works
Coffee consumption and health: umbrella review of meta-analyses of multiple health outcomes
Poole, R., Kennedy, O. J., Roderick, P., Fallowfield, J. A., Hayes, P. C., & Parkes, J. (2017) · BMJ · doi:10.1136/bmj.j5024 · Cited by 835 · see citing works
A systematic review of reviews on the prevalence of anxiety disorders in adult populations
Remes, O., Brayne, C., van der Linde, R., & Lafortune, L. (2016) · Brain and Behavior · doi:10.1002/brb3.497 · Cited by 802 · see citing works
Borah, R., Brown, A. W., Capers, P. L., & Kaiser, K. A. (2017) · BMJ Open · doi:10.1136/bmjopen-2016-012545 · Cited by 737 · see citing works
Hong, Q. N., Pluye, P., Bujold, M., & Wassef, M. (2017) · Systematic Reviews · doi:10.1186/s13643-017-0454-2 · Cited by 754 · see citing works
“Also, while some tools for appraising systematic reviews of quantitative studies exist [19, 20], to our know-”
BackgroundStubbs, B., Vancampfort, D., Hallgren, M., Firth, J., Veronese, N., Solmi, M., Brand, S., Cordes, J., Malchow, B., Gerber, M., et al. (2018) · European Psychiatry · doi:10.1016/j.eurpsy.2018.07.004 · Cited by 697 · see citing works
How to write a systematic review
Harris, J. D., Quatman, C. E., Manring, M. M., Siston, R. A., & Flanigan, D. C. (2014) · The American Journal of Sports Medicine · doi:10.1177/0363546513497567 · Cited by 606 · see citing works
Latulippe, K., Hamel, C., & Giroux, D. (2017) · Journal of Medical Internet Research · doi:10.2196/jmir.6731 · Cited by 458 · see citing works
Self-citationPrevention of Psychosis: Advances in Detection, Prognosis, and Intervention
Fusar-Poli, P., Salazar de Pablo, G., Correll, C. U., Meyer-Lindenberg, A., Millan, M. J., Borgwardt, S., Galderisi, S., Bechdolf, A., Pfennig, A., Kessing, L. V., et al. (2020) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2019.4779 · Cited by 476 · see citing works
Klem, F., Wadhwa, A., Prokop, L. J., Sundt, W. J., Farrugia, G., Camilleri, M., Singh, S., & Grover, M. (2017) · Gastroenterology · doi:10.1053/j.gastro.2016.12.039 · Cited by 474 · see citing works
Leucht, S., Hierl, S., Kissling, W., Dold, M., & Davis, J. M. (2012) · British Journal of Psychiatry · doi:10.1192/bjp.bp.111.096594 · Cited by 451 · see citing works
Ashdown-Franks, G., Firth, J., Carney, R., Carvalho, A. F., Hallgren, M., Koyanagi, A., Rosenbaum, S., Schuch, F. B., Smith, L., Solmi, M., et al. (2020) · Sports Medicine · doi:10.1007/s40279-019-01187-6 · Cited by 446 · see citing works
Online Prevention Aimed at Lifestyle Behaviors: A Systematic Review of Reviews
Kohl, L. F., Crutzen, R., & de Vries, N. K. (2013) · Journal of Medical Internet Research · doi:10.2196/jmir.2665 · Cited by 388 · see citing works
Systematic meta-review and quality assessment of the structural brain alterations in schizophrenia
Shepherd, A. M., Laurens, K. R., Matheson, S. L., Carr, V. J., & Green, M. J. (2012) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2011.12.015 · Cited by 412 · see citing works
A meta-review of stress, coping and interventions in dementia and dementia caregiving
Gilhooly, K. J., Gilhooly, M. L. M., Sullivan, M. P., McIntyre, A., Wilson, L., Harding, E., Woodbridge, R., & Crutch, S. (2016) · BMC Geriatrics · doi:10.1186/s12877-016-0280-8 · Cited by 410 · see citing works
Tate, R. L., Perdices, M., Rosenkoetter, U., Wakim, D., Godbee, K., Togher, L., & McDonald, S. (2013) · Neuropsychological Rehabilitation · doi:10.1080/09602011.2013.824383 · Cited by 358 · see citing works
Hennessy, E. A., & Johnson, B. T. (2020) · Research Synthesis Methods · doi:10.1002/jrsm.1390 · Cited by 348 · see citing works
Hersi, M., Irvine, B., Gupta, P., Gomes, J., Birkett, N., & Krewski, D. (2017) · NeuroToxicology · doi:10.1016/j.neuro.2017.03.006 · Cited by 332 · see citing works
Huhn, M., Tardy, M., Spineli, L. M., Kissling, W., Förstl, H., Pitschel-Walz, G., Leucht, C., Samara, M., Dold, M., Davis, J. M., et al. (2014) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2014.112 · Cited by 333 · see citing works
Systematic review adherence to methodological or reporting quality
Pussegoda, K., Turner, L., Garritty, C., Mayhew, A., Skidmore, B., Stevens, A., Boutron, I., Sarkis-Onofre, R., Bjerre, L. M., Hróbjartsson, A., et al. (2017) · Systematic Reviews · doi:10.1186/s13643-017-0527-2 · Cited by 322 · see citing works
Shaw, L. M., Arias, J., Blennow, K., Galasko, D., Molinuevo, J. L., Salloway, S., Schindler, S., Carrillo, M. C., Hendrix, J. A., Ross, A., et al. (2018) · Alzheimer's & Dementia · doi:10.1016/j.jalz.2018.07.220 · Cited by 266 · see citing works
Exercise and sleep: a systematic review of previous meta‐analyses
Kelley, G. A., & Kelley, K. S. (2017) · Journal of Evidence-Based Medicine · doi:10.1111/jebm.12236 · Cited by 258 · see citing works
Meta‐analysis of meta‐analyses of anterior cruciate ligament injury reduction training programs
Webster, K. E., & Hewett, T. E. (2018) · Journal of Orthopaedic Research · doi:10.1002/jor.24043 · Cited by 253 · see citing works
Integrated care programmes for adults with chronic conditions: a meta-review
Martínez-González, N. A., Berchtold, P., Ullman, K., Busato, A., & Egger, M. (2014) · International Journal for Quality in Health Care · doi:10.1093/intqhc/mzu071 · Cited by 253 · see citing works
Advance care planning in dementia: recommendations for healthcare professionals.
Piers, R., Albers, G., Gilissen, J., De Lepeleire, J., Steyaert, J., Van Mechelen, W., Steeman, E., Dillen, L., Vanden Berghe, P., & Van den Block, L. (2018) · BMC Palliative Care · doi:10.1186/s12904-018-0332-2 · Cited by 254 · see citing works
The methodological and reporting quality of systematic reviews from China and the USA are similar
Tian, J., Zhang, J., Ge, L., Yang, K., & Song, F. (2017) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2016.12.004 · Cited by 248 · see citing works
Rouleau, G., Gagnon, M.-P., Côté, J., Payne-Gagnon, J., Hudson, E., & Dubois, C.-A. (2017) · Journal of Medical Internet Research · doi:10.2196/jmir.6686 · Cited by 244 · see citing works
“The three reviewers (GR, JPG, and EH) were involved in the methodological quality assessment of the reviews that met the eligibility criteria, using the assessment of multiple systematic reviews (AMSTAR) tool [33,34].”
Patnode, C. D., Henderson, J. T., Thompson, J. H., Senger, C. A., Fortmann, S. P., & Whitlock, E. P. (2015) · Annals of Internal Medicine · doi:10.7326/m15-0171 · Cited by 238 · see citing works
Odgers‐Jewell, K., Ball, L. E., Kelly, J. T., Isenring, E. A., Reidlinger, D. P., & Thomas, R. (2017) · Diabetic Medicine · doi:10.1111/dme.13340 · Cited by 224 · see citing works
Vancampfort, D., Firth, J., Correll, C. U., Solmi, M., Siskind, D., De Hert, M., Carney, R., Koyanagi, A., Carvalho, A. F., Gaughran, F., et al. (2019) · World Psychiatry · doi:10.1002/wps.20614 · Cited by 227 · see citing works
Evidence-based umbrella review of 162 peripheral biomarkers for major mental disorders
Carvalho, A. F., Solmi, M., Sanches, M., Machado, M. O., Stubbs, B., Ajnakina, O., Sherman, C., Sun, Y. R., Liu, C. S., Brunoni, A. R., et al. (2020) · Translational Psychiatry · doi:10.1038/s41398-020-0835-5 · Cited by 204 · see citing works
Correll, C. U., Rubio, J. M., Inczedy-Farkas, G., Birnbaum, M. L., Kane, J. M., & Leucht, S. (2017) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2017.0624 · Cited by 214 · see citing works
Overviews of reviews often have limited rigor: a systematic review
Pieper, D., Buechter, R., Jerinic, P., & Eikermann, M. (2012) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2012.06.015 · Cited by 213 · see citing works
Cooper, H., & Koenka, A. C. (2012) · American Psychologist · doi:10.1037/a0027119 · Cited by 196 · see citing works
Network meta-analyses should be the highest level of evidence in treatment guidelines
Leucht, S., Chaimani, A., Cipriani, A. S., Davis, J. M., Furukawa, T. A., & Salanti, G. (2016) · European Archives of Psychiatry and Clinical Neuroscience · doi:10.1007/s00406-016-0715-4 · Cited by 202 · see citing works
Vukušić Rukavina, T., Viskić, J., Machala Poplašen, L., Relić, D., Marelić, M., Jokic, D., & Sedak, K. (2021) · Journal of Medical Internet Research · doi:10.2196/25770 · Cited by 199 · see citing works
Mbuagbaw, L., Mursleen, S., Lytvyn, L., Smieja, M., Dolovich, L., & Thabane, L. (2015) · BMC Health Services Research · doi:10.1186/s12913-014-0654-6 · Cited by 187 · see citing works
Umbrella reviews: what they are and why we need them
Papatheodorou, S. (2019) · European Journal of Epidemiology · doi:10.1007/s10654-019-00505-6 · Cited by 201 · see citing works
Pollock, M., Fernandes, R. M., Becker, L. A., Featherstone, R., & Hartling, L. (2016) · Systematic Reviews · doi:10.1186/s13643-016-0367-5 · Cited by 195 · see citing works
Gartlehner, G., Wagner, G., Matyas, N., Titscher, V., Greimel, J., Lux, L., Gaynes, B. N., Viswanathan, M., Patel, S., & Lohr, K. N. (2017) · BMJ Open · doi:10.1136/bmjopen-2016-014912 · Cited by 192 · see citing works
Templier, M., & Paré, G. (2018) · European Journal of Information Systems · doi:10.1080/0960085x.2017.1398880 · Cited by 180 · see citing works
Adherence influencing factors – a systematic review of systematic reviews
Mathes, T., Jaschinski, T., & Pieper, D. (2014) · Archives of Public Health · doi:10.1186/2049-3258-72-37 · Cited by 183 · see citing works
Smith, K. E., Mason, T. B., Johnson, J. S., Lavender, J. M., & Wonderlich, S. A. (2018) · International Journal of Eating Disorders · doi:10.1002/eat.22929 · Cited by 170 · see citing works
Pandor, A., Thokala, P., Gomersall, T., Baalbaki, H., Stevens, J., Wang, J., Wong, R., Brennan, A., & Fitzgerald, P. (2013) · Health Technol Assess · doi:10.3310/hta17320 · Cited by 159 · see citing works
Treatments for women with gestational diabetes mellitus: an overview of Cochrane systematic reviews
Martis, R., Crowther, C. A., Shepherd, E., Alsweiler, J., Downie, M. R., & Brown, J. (2018) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd012327.pub2 · Cited by 159 · see citing works
Preconditions for successful advance care planning in nursing homes: A systematic review.
Gilissen, J., Pivodic, L., Smets, T., Gastmans, C., Vander Stichele, R., Deliens, L., & Van den Block, L. (2017) · International Journal of Nursing Studies · doi:10.1016/j.ijnurstu.2016.12.003 · Cited by 156 · see citing works
Häuser, W., Schug, S., & Furlan, A. D. (2017) · PAIN Reports · doi:10.1097/pr9.0000000000000599 · Cited by 154 · see citing works
Lunny, C., Brennan, S. E., McDonald, S., & McKenzie, J. E. (2018) · Systematic Reviews · doi:10.1186/s13643-018-0784-8 · Cited by 139 · see citing works
Puthussery, S., Chutiyami, M., Tseng, P.-C., Kilby, L., & Kapadia, J. (2018) · BMC Pediatrics · doi:10.1186/s12887-018-1205-9 · Cited by 140 · see citing works
Overviews of systematic reviews: great promise, greater challenge
McKenzie, J. E., & Brennan, S. E. (2017) · Systematic Reviews · doi:10.1186/s13643-017-0582-8 · Cited by 134 · see citing works
Assistive Technologies in Dementia Care: An Updated Analysis of the Literature
Pappadà, A., Chattat, R., Chirico, I., Valente, M., & Ottoboni, G. (2021) · Frontiers in Psychology · doi:10.3389/fpsyg.2021.644587 · Cited by 131 · see citing works
Tricco, A. C., Zarin, W., Ghassemi, M., Nincic, V., Lillie, E., Page, M. J., Shamseer, L., Antony, J., Rios, P., Hwee, J., et al. (2018) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2017.10.014 · Cited by 122 · see citing works
Worswick, J., Wayne, S. C., Bennett, R., Fiander, M., Mayhew, A., Weir, M. C., Sullivan, K. J., & Grimshaw, J. M. (2013) · Systematic Reviews · doi:10.1186/2046-4053-2-26 · Cited by 130 · see citing works
Self-citationA comparison of two assessment tools used in overviews of systematic reviews: ROBIS versus AMSTAR-2
Perry, R., Whitmarsh, A., Leach, V., & Davies, P. (2021) · Systematic Reviews · doi:10.1186/s13643-021-01819-x · Cited by 130 · see citing works
Jayedi, A., & Shab-Bidar, S. (2020) · Advances in Nutrition · doi:10.1093/advances/nmaa029 · Cited by 124 · see citing works
Weight loss and serum lipids in overweight and obese adults: A systematic review and meta-analysis
Hasan, B., Nayfeh, T., Alzuabi, M., Wang, Z., Kuchkuntla, A. R., Prokop, L. J., Newman, C. B., Murad, M. H., & Rajjo, T. I. (2020) · The Journal of Clinical Endocrinology & Metabolism · doi:10.1210/clinem/dgaa673 · Cited by 126 · see citing works
Tools to support evidence-informed public health decision making
Yost, J., Dobbins, M., Traynor, R., DeCorby, K., Workentine, S., & Greco, L. (2014) · BMC Public Health · doi:10.1186/1471-2458-14-728 · Cited by 116 · see citing works
Neel, M. L. M., Stark, A. R., & Maitre, N. L. (2018) · Child: Care, Health and Development · doi:10.1111/cch.12561 · Cited by 114 · see citing works
Allen, M. S., & Walter, E. E. (2019) · The Journal of Sexual Medicine · doi:10.1016/j.jsxm.2019.01.314 · Cited by 116 · see citing works
Prothero, L., Barley, E., Galloway, J., Georgopoulou, S., & Sturt, J. (2018) · International Journal of Nursing Studies · doi:10.1016/j.ijnurstu.2018.03.008 · Cited by 115 · see citing works
Jones, K. R., Lekhak, N., & Kaewluang, N. (2014) · Worldviews on Evidence-Based Nursing · doi:10.1111/wvn.12030 · Cited by 114 · see citing works
Cholesterol and Alzheimer’s Disease Risk: A Meta-Meta-Analysis
Sáiz-Vazquez, O., Puente-Martínez, A., Ubillos-Landa, S., Pacheco-Bonrostro, J., & Santabárbara, J. (2020) · Brain Sciences · doi:10.3390/brainsci10060386 · Cited by 135 · see citing works
Preventive Treatments for Psychosis: Umbrella Review (Just the Evidence)
Fusar-Poli, P., Davies, C., Solmi, M., Brondino, N., De Micheli, A., Kotlicka-Antczak, M., Shin, J. I., & Radua, J. (2019) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2019.00764 · Cited by 108 · see citing works
The Efficacy, Safety and Applications of Medical Hypnosis: A Systematic Review of Meta-analyses
Häuser, W., Hagl, M., Schmierer, A., & Hansen, E. (2016) · Deutsches Ärzteblatt international · doi:10.3238/arztebl.2016.0289 · Cited by 110 · see citing works
Is Concurrent Training Efficacious Antihypertensive Therapy? A Meta-Analysis
CORSO, L. M. L., MACDONALD, H. V., JOHNSON, B. T., FARINATTI, P., LIVINGSTON, J., ZALESKI, A. L., BLANCHARD, A., & PESCATELLO, L. S. (2016) · Medicine & Science in Sports & Exercise · doi:10.1249/mss.0000000000001056 · Cited by 107 · see citing works
Yan, P., Yao, L., Li, H., Zhang, M., Xun, Y., Li, M., Cai, H., Lu, C., Hu, L., Guo, T., et al. (2019) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2018.12.013 · Cited by 107 · see citing works
The health and wellbeing needs of veterans: a rapid review
Oster, C., Morello, A., Venning, A., Redpath, P., & Lawn, S. (2017) · BMC Psychiatry · doi:10.1186/s12888-017-1547-0 · Cited by 98 · see citing works
Vancampfort, D., Stubbs, B., Van Damme, T., Smith, L., Hallgren, M., Schuch, F., Deenik, J., Rosenbaum, S., Ashdown-Franks, G., Mugisha, J., et al. (2021) · Journal of Psychiatric Research · doi:10.1016/j.jpsychires.2020.12.048 · Cited by 99 · see citing works
Czosnek, L., Lederman, O., Cormie, P., Zopf, E., Stubbs, B., & Rosenbaum, S. (2019) · Mental Health and Physical Activity · doi:10.1016/j.mhpa.2018.11.001 · Cited by 94 · see citing works
Kotsakis, G. A., Lian, Q., Ioannou, A. L., Michalowicz, B. S., John, M. T., & Chu, H. (2018) · Journal of Periodontology · doi:10.1002/jper.17-0368 · Cited by 97 · see citing works
Resource use during systematic review production varies widely: a scoping review
Nussbaumer-Streit, B., Ellen, M., Klerings, I., Sfetcu, R., Riva, N., Mahmić-Kaknjo, M., Poulentzas, G., Martinez, P., Baladia, E., Ziganshina, L. E., et al. (2021) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2021.05.019 · Cited by 98 · see citing works
What we already know about the effects of exercise in patients with fibromyalgia: An umbrella review
Andrade, A., Dominski, F. H., & Sieczkowska, S. M. (2020) · Seminars in Arthritis and Rheumatism · doi:10.1016/j.semarthrit.2020.02.003 · Cited by 95 · see citing works
Lunny, C., Brennan, S. E., Reid, J., McDonald, S., & McKenzie, J. E. (2020) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2019.09.025 · Cited by 71 · see citing works
Quality of systematic reviews in pediatric oncology – A systematic review
Lundh, A., Knijnenburg, S. L., Jørgensen, A. W., van Dalen, E. C., & Kremer, L. C. M. (2009) · Cancer Treatment Reviews · doi:10.1016/j.ctrv.2009.08.010 · Cited by 78 · see citing works
Drukker, M., Weltens, I., van Hooijdonk, C. F. M., Vandenberk, E., & Bak, M. (2021) · Frontiers in Research Metrics and Analytics · doi:10.3389/frma.2021.675071 · Cited by 75 · see citing works
Fit for purpose: perspectives on rapid reviews from end-user interviews
Hartling, L., Guise, J.-M., Hempel, S., Featherstone, R., Mitchell, M. D., Motu’apuaka, M. L., Robinson, K. A., Schoelles, K., Totten, A., Whitlock, E., et al. (2017) · Systematic Reviews · doi:10.1186/s13643-017-0425-7 · Cited by 65 · see citing works
Tafelski, S., Häuser, W., & Schäfer, M. (2016) · Der Schmerz · doi:10.1007/s00482-015-0092-3 · Cited by 71 · see citing works
The reliability of evidence review methodology in environmental science and conservation
O’Leary, B. C., Kvist, K., Bayliss, H. R., Derroire, G., Healey, J. R., Hughes, K., Kleinschroth, F., Sciberras, M., Woodcock, P., & Pullin, A. S. (2016) · Environmental Science & Policy · doi:10.1016/j.envsci.2016.06.012 · Cited by 65 · see citing works
The effectiveness of mindfulness-based interventions on anxiety disorders. A systematic meta-review
Fumero, A., Peñate, W., Oyanadel, C., & Porter, B. (2020) · European Journal of Investigation in Health, Psychology and Education · doi:10.3390/ejihpe10030052 · Cited by 64 · see citing works
“The methodological quality of these reviews was examined using the AMSTAR tool [38].”
MethodRuane, A., & Carr, A. (2019) · Family Process · doi:10.1111/famp.12352 · Cited by 59 · see citing works
Pedro, J., Monteiro‐Reis, S., Carvalho‐Maia, C., Henrique, R., Jerónimo, C., & Silva, E. R. (2021) · Psycho-Oncology · doi:10.1002/pon.5771 · Cited by 58 · see citing works
Developing WHO rapid advice guidelines in the setting of a public health emergency
Garritty, C. M., Norris, S. L., & Moher, D. (2017) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2016.08.010 · Cited by 56 · see citing works
Effectiveness of multidisciplinary programmes for clinical pain conditions: an umbrella review.
Dragioti, E., Evangelou, E., Larsson, B., & Gerdle, B. (2018) · Journal of Rehabilitation Medicine · doi:10.2340/16501977-2377 · Cited by 53 · see citing works
Gu, D.-T., Tung, T.-H., Jiesisibieke, Z. L., Chien, C.-W., & Liu, W.-Y. (2022) · Frontiers in Pharmacology · doi:10.3389/fphar.2021.790901 · Cited by 54 · see citing works
Silva, M. R. O. B. D., Moura, Y. A. S., Converti, A., Porto, A. L. F., Viana Marques, D. D. A., & Bezerra, R. P. (2021) · Algal Research · doi:10.1016/j.algal.2021.102396 · Cited by 52 · see citing works
Šouláková, B., Kasal, A., Butzer, B., & Winkler, P. (2019) · The Journal of Primary Prevention · doi:10.1007/s10935-019-00552-5 · Cited by 49 · see citing works
Ho, R. S., Wu, X., Yuan, J., Liu, S., Lai, X., Wong, S. Y., & Chung, V. C. (2015) · npj Primary Care Respiratory Medicine · doi:10.1038/npjpcrm.2014.102 · Cited by 50 · see citing works
Gómez-García, F., Ruano, J., Gay-Mimbrera, J., Aguilar-Luque, M., Sanz-Cabanillas, J. L., Alcalde-Mellado, P., Maestre-López, B., Carmona-Fernández, P. J., González-Padilla, M., García-Nieto, A. V., et al. (2017) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2017.08.015 · Cited by 37 · see citing works
Solmi, M., Correll, C. U., Carvalho, A. F., & Ioannidis, J. P. A. (2018) · Epidemiology and Psychiatric Sciences · doi:10.1017/s204579601800032x · Cited by 44 · see citing works
Chou, R., Selph, S., Dana, T., Bougatsos, C., Zakher, B., Blazina, I., & Korthuis, P. T. (2012) · Annals of Internal Medicine · doi:10.7326/0003-4819-157-10-201211200-00007 · Cited by 46 · see citing works
- Authors
- Shea, B. J., Hamel, C., Wells, G. A., Bouter, L. M., Kristjansson, E., Grimshaw, J., Henry, D. A., & Boers, M.
- Affiliations
- Amsterdam Public Health (NL)
- Institute of Population and Public Health (CA)
- University of Ottawa (CA)
- Institute for Clinical Evaluative Sciences (CA)
- Amsterdam UMC Location Vrije Universiteit Amsterdam (NL)
- Corresponding author
- Shea, Beverley J.
- ORCID
- Venue
- Journal of Clinical Epidemiology
- Publisher
- Elsevier BV
- Published
- 2009-10
- Volume
- 62
- Issue
- 10
- Pages
- 1013-1020
- ISSN
- 0895-4356 (ISSN-L)
- OpenAlex
- W2073644162
- Cited by
- 1,755
- References
- 47
- Citations by year
- 2026: 47 · 2025: 58 · 2024: 64 · 2023: 100 · 2022: 79 · 2021: 148 · 2020: 140 · 2019: 158 · 2018: 168 · 2017: 216 · 2016: 149 · 2015: 121 · 2014: 108 · 2013: 83 · 2012: 49
- FWCI
- 2.87
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Journal of Clinical Epidemiology (publishedVersion)
- PubMed (publishedVersion)
- Pure Amsterdam UMC (publishedVersion)
- Amsterdam UMC (VU Amsterdam) - Institutional Repository (publishedVersion)
- VU Research Portal (publishedVersion, other-oa)
- Digital Academic REpository of VU University Amsterdam (Vrije Universiteit Amsterdam) (submittedVersion)
- Data Archiving and Networked Services (DANS) (submittedVersion)
- Data Archiving and Networked Services (DANS) (submittedVersion, other-oa)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Sociology and Political Science › Delphi Technique in Research
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Reliability and Agreement in Measurement
- Keywords
- AMSTAR, methodological quality assessment, systematic reviews, inter-rater reliability, construct validity
- MeSH terms
- Feasibility Studies, Humans, Quality Control, Research Design, Review Literature as Topic, Reproducibility of Results, Observer Variation, Evidence-Based Medicine
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
