Adjustment of meta-analyses on the basis of quality scores should be abandoned

Herbison, Peter; Hay-Smith, Jean; Gillespie, William J.

doi:10.1016/j.jclinepi.2006.03.008

Cited by 207Meta-analysis and systematic reviewsDelphi Technique in ResearchStatistical Methods in Clinical Trials

Adjustment of meta-analyses on the basis of quality scores should be abandoned

Herbison, Peter, Hay-Smith, Jean, Gillespie, William J.·2006·Journal of Clinical Epidemiology

DOI 10.1016/j.jclinepi.2006.03.008

Open this paper in CoChat — free account

Abstract

Objective To find if a particular quality score was better than others at validly scoring the quality of randomized controlled trials, both by examining the consistency of dividing studies into high and low quality and using a large study as a reference standard.Study design and setting Observational study of meta-analyses from the Cochrane Library. These had to have binary outcomes that included more than 10 studies, one or more of which randomized more than 500 people into each group.Results Eighteen systematic reviews, with 65 meta-analyses using binary outcomes, were included and the included trials were scored for 43 different quality scores. None of these scores was better at dividing the studies in to low and high quality, and none of the scores was better over the 65 meta-analyses in making the result closer to the reference standard.Conclusion None of the quality scores found appeared to measure quality validly. It is a mistake to assign meaning to the result of a quality score.

Integrity signals

Registry

Crossref · 10.1016/j.jclinepi.2006.03.008

Cites

73 of 73 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

The study performed three complementary quality and risk-of-bias analyses.
“Recognising that quality score use in meta-analyses remains controversial,12,13 we did three complementary quality and risk of bias analyses.”
From The effect of a reduction in alcohol consumption on blood pressure: a systematic review and meta-analysis (2017) · cites this paper as [13]

3 retracted

Antibiotics for acute maxillary sinusitis
Ahovuo-Saloranta, A., Rautakorpi, U.-M., Borisenko, O., Liira, H., Williams, J. W., & Mākelä, M. (2008) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd000243.pub2 · Cited by 241 · see citing works
Retracted
Prostaglandins versus oxytocin for prelabour rupture of membranes at or near term
Tan, B. P., & Hannah, M. (1997) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd000158 · Cited by 28 · see citing works
Retracted
Prostaglandins for prelabour rupture of membranes at or near term
Tan, B. P., & Hannah, M. (1997) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd000178 · Cited by 20 · see citing works
Retracted
Assessing the quality of reports of randomized clinical trials: Is blinding necessary?
Jadad, A. R., Moore, R. A., Carroll, D., Jenkinson, C., Reynolds, D. J. M., Gavaghan, D. J., & McQuay, H. J. (1996) · Controlled Clinical Trials · doi:10.1016/0197-2456(95)00134-4 · Cited by 17,582 · see citing works
The feasibility of creating a checklist for the assessment of the methodological quality both of randomised and non-randomised studies of health care interventions.
Downs, S. H., & Black, N. (1998) · Journal of Epidemiology and Community Health · doi:10.1136/jech.52.6.377 · Cited by 8,252 · see citing works
Untitled work
Schulz (1995) · doi:10.1001/jama.1995.03520290060030 · Cited by 5,634 · see citing works
Does quality of reports of randomised trials affect estimates of intervention efficacy reported in meta-analyses?
Moher, D., Pham, B., Jones, A., Cook, D. J., Jadad, A. R., Moher, M., Tugwell, P., & Klassen, T. P. (1998) · The Lancet · doi:10.1016/s0140-6736(98)01085-x · Cited by 3,560 · see citing works
Nicotine replacement therapy for smoking cessation
Stead, L. F., Perera, R., Bullen, C., Mant, D., Hartmann‐Boyce, J., Cahill, K., & Lancaster, T. (2012) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd000146.pub4 · Cited by 2,514 · see citing works
The Delphi list: a criteria list for quality assessment of randomized clinical trials for conducting systematic reviews developed by Delphi consensus.
Verhagen, A., de Vet, H. D., de Bie, R. D., Kessels, A., Boers, M., Bouter, L., & Knipschild, P. (1998) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(98)00131-0 · Cited by 2,473 · see citing works
Updated Method Guidelines for Systematic Reviews in the Cochrane Collaboration Back Review Group
van Tulder, M., Furlan, A., Bombardier, C., & Bouter, L. (2003) · Spine · doi:10.1097/01.brs.0000065484.95996.af · Cited by 2,180 · see citing works
The Hazards of Scoring the Quality of Clinical Trials for Meta-analysis
Jüni, P. (1999) · JAMA · doi:10.1001/jama.282.11.1054 · Cited by 1,981 · see citing works
A method for assessing the quality of a randomized control trial
Chalmers, T. C., Smith, H., Blackburn, B., Silverman, B., Schroeder, B., Reitman, D., & Ambroz, A. (1981) · Controlled Clinical Trials · doi:10.1016/0197-2456(81)90056-8 · Cited by 1,349 · see citing works
Assessing the quality of randomized controlled trials: An annotated bibliography of scales and checklists
Moher, D., Jadad, A. R., Nichol, G., Penman, M., Tugwell, P., & Walsh, S. (1995) · Controlled Clinical Trials · doi:10.1016/0197-2456(94)00031-w · Cited by 1,308 · see citing works
Discrepancies between Meta-Analyses and Subsequent Large Randomized, Controlled Trials
LeLorier, J., Grégoire, G., Benhaddad, A. A., Lapierre, J., & Derderian, F. (1997) · New England Journal of Medicine · doi:10.1056/nejm199708213370806 · Cited by 1,171 · see citing works
Are the clinical effects of homoeopathy placebo effects? A meta-analysis of placebo-controlled trials
Linde, K., Clausius, N., Ramirez, G., Melchart, D., Eitel, F., Hedges, L. V., & Jonas, W. B. (1997) · The Lancet · doi:10.1016/s0140-6736(97)02293-9 · Cited by 979 · see citing works
Low-molecular-weight heparin versus standard heparin in general and orthopaedic surgery: a meta-analysis
Nurmohamed, M. T., Buller, H. R., Dekker, E., Hommes, D. W., Rosendaal, F. R., Briët, E., Rosendaal, F. R., & Vandenbroucke, J. P. (1992) · The Lancet · doi:10.1016/0140-6736(92)93223-a · Cited by 664 · see citing works
Incorporating variations in the quality of individual randomized trials into meta-analysis
Detsky, A. S., Naylor, C. D., O'Rourke, K., McGeer, A. J., & L'Abbé, K. A. (1992) · Journal of Clinical Epidemiology · doi:10.1016/0895-4356(92)90085-2 · Cited by 640 · see citing works
Clinical trials of homoeopathy.
Kleijnen, J., Knipschild, P. G., & ter Riet, G. (1991) · BMJ · doi:10.1136/bmj.302.6772.316 · Cited by 604 · see citing works
Effects of Intensity of Rehabilitation After Stroke
Kwakkel, G., WAGENAAR, R. C., Koelman, T. W., Lankhorst, G. J., & Koetsier, J. C. (1997) · Stroke · doi:10.1161/01.str.28.8.1550 · Cited by 588 · see citing works
Antidepressant-induced analgesia in chronic non-malignant pain: a meta-analysis of 39 placebo-controlled studies
Onghena, P., & Houdenhove, B. V. (1992) · Pain · doi:10.1016/0304-3959(92)90144-z · Cited by 560 · see citing works
Total Parenteral Nutrition in the Critically Ill Patient
Heyland, D. K., MacDonald, S., Keefe, L., & Drover, J. (1998) · JAMA · doi:10.1001/jama.280.23.2013 · Cited by 532 · see citing works
How study design affects outcomes in comparisons of therapy. I: Medical
Colditz, G. A., Miller, J. N., & Mosteller, F. C. (1989) · Statistics in Medicine · doi:10.1002/sim.4780080408 · Cited by 520 · see citing works
Invited commentary: a critical look at some popular meta-analytic methods
Greenland, S. (1994) · American Journal of Epidemiology · doi:10.1093/oxfordjournals.aje.a117248 · Cited by 511 · see citing works
Corticosteroid injections for lateral epicondylitis: a systematic review
Smidt, N., Assendelft, W. J., van der Windt, D., Hay, E. M., Buchbinder, R., & Bouter, L. M. (2002) · Pain · doi:10.1016/s0304-3959(01)00388-8 · Cited by 431 · see citing works
Quality of Reporting of Randomized Trials as a Measure of Methodologic Quality
Huwiler-Müntener, K. (2002) · JAMA · doi:10.1001/jama.287.21.2801 · Cited by 397 · see citing works
On the bias produced by quality scores in meta-analysis, and a hierarchical view of proposed solutions
Greenland, S. (2001) · Biostatistics · doi:10.1093/biostatistics/2.4.463 · Cited by 359 · see citing works
Methodology and overt and hidden bias in reports of 196 double-blind trials of nonsteroidal antiinflammatory drugs in rheumatoid arthritis
Gøtzsche, P. C. (1989) · Controlled Clinical Trials · doi:10.1016/0197-2456(89)90017-2 · Cited by 339 · see citing works
Large Trials vs Meta-analysis of Smaller Trials: How Do Their Results Compare?
Cappelleri, J. C. (1996) · JAMA · doi:10.1001/jama.1996.03540160054033 · Cited by 336 · see citing works
Manuscript Quality before and after Peer Review and Editing at <i>Annals of Internal Medicine</i>
Goodman, S. N., Berlin, J., Fletcher, S. W., & Fletcher, R. H. (1994) · Annals of Internal Medicine · doi:10.7326/0003-4819-121-1-199407010-00003 · Cited by 324 · see citing works
Guide to clinical trials
Spilker, B. (1991) · Cited by 322 · see citing works
Acupuncture and chronic pain: A criteria-based meta-analysis
ter Riet, G., Kleunen, J., & Knipschild, P. G. (1990) · Journal of Clinical Epidemiology · doi:10.1016/0895-4356(90)90020-p · Cited by 314 · see citing works
Intravenous immunoglobulin for treating sepsis, severe sepsis and septic shock
Alejandria, M. M., Lansang, M. A., Dans, L. F., & Mantaring, J. B. (2002) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd001090 · Cited by 313 · see citing works
Bad reporting does not mean bad methods for randomised trials: observational study of randomised controlled trials performed by the Radiation Therapy Oncology Group
Soares, H. P., Daniels, S., Kumar, A., Clarke, M. J., Scott, C., Swann, S., & Djulbegović, B. (2004) · BMJ · doi:10.1136/bmj.328.7430.22 · Cited by 310 · see citing works
Respiratory Muscle Training in Chronic Airflow Limitation: A Meta-Analysis
Smith, K., Cook, D., Guyatt, G. H., Madhavan, J., & Oxman, A. D. (1992) · American Review of Respiratory Disease · doi:10.1164/ajrccm/145.3.533 · Cited by 274 · see citing works
Spinal manipulation and mobilisation for back and neck pain: a blinded review.
Koes, B. W., Assendelft, W. J., van der Heijden, G. J. M. G., Bouter, L. M., & Knipschild, P. G. (1991) · BMJ · doi:10.1136/bmj.303.6813.1298 · Cited by 270 · see citing works
Teasing apart quality and validity in systematic reviews: an example from acupuncture trials in chronic neck and back pain
Smith, L. A., Oldman, A., McQuay, H. J., & Moore, A. R. (2000) · Pain · doi:10.1016/s0304-3959(00)00234-7 · Cited by 249 · see citing works
The art of quality assessment of RCTs included in systematic reviews
Verhagen, A. P., de Vet, H. C. W., de Bie, R. A., Boers, M., & van den Brandt, P. A. (2001) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(00)00360-7 · Cited by 237 · see citing works
Do Corticosteroids Reduce Mortality from Alcoholic Hepatitis?
Imperiale, T. F., & McCullough, A. J. (1990) · Annals of Internal Medicine · doi:10.7326/0003-4819-113-4-299 · Cited by 231 · see citing works
A systematic review of the relationship between overjet size and traumatic dental injuries
Nguyen, Q. (1999) · The European Journal of Orthodontics · doi:10.1093/ejo/21.5.503 · Cited by 224 · see citing works
Predictive ability of meta-analyses of randomised controlled trials
Villar, J., Carroli, G., & Belizán, J. M. (1995) · The Lancet · doi:10.1016/s0140-6736(95)90646-0 · Cited by 222 · see citing works
A cohort study of summary reports of controlled trials.
Chalmers, I., Adams, M., Dickersin, K., Hetherington, J., Tarnow-Mordi, W., Meinert, C., Tonascia, S., & Chalmers, T. (1990) · Journal of the American Medical Association (JAMA) · doi:10.1001/jama.1990.03440100117017 · Cited by 200 · see citing works
Impact of Study Quality on Outcome in Placebo-Controlled Trials of Homeopathy
Linde, K., Scholz, M., Ramı́rez, G., Clausius, N., Melchart, D., & Jonas, W. B. (1999) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(99)00048-7 · Cited by 200 · see citing works
Instruments for Assessing the Quality of Drug Studies Published in the Medical Literature
Cho, M. K. (1994) · JAMA · doi:10.1001/jama.1994.03520020027007 · Cited by 180 · see citing works
Where now for meta-analysis?
Egger, M., Ebrahim, S., & Smith, G. D. (2002) · International Journal of Epidemiology · doi:10.1093/ije/31.1.1 · Cited by 160 · see citing works
Discordance between Meta-analyses and Large-Scale Randomized, Controlled Trials: Examples from the Management of Acute Myocardial Infarction
Borzak, S. L., & Ridker, P. M. (1995) · Annals of Internal Medicine · doi:10.7326/0003-4819-123-11-199512010-00010 · Cited by 131 · see citing works
Aid to the Evaluation of Therapeutic Studies
REISCH, J. S., Tyson, J. E., & Mize, S. G. (1989) · PEDIATRICS · doi:10.1542/peds.84.5.815 · Cited by 123 · see citing works
Ten‐year followup study of patients with yersinia arthritis
Leirisalo‐Repo, M., & Suoranta, H. (1988) · Arthritis & Rheumatism · doi:10.1002/art.1780310410 · Cited by 116 · see citing works
Calcium antagonists for acute ischemic stroke
Horn, J., & Limburg, M. (2000) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd001928 · Cited by 101 · see citing works
Maximal androgen blockade for advanced prostate cancer
Schmitt, B. P., Bennett, C. L., Seidenfeld, J., Samson, D. J., & Wilt, T. J. (1999) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd001526 · Cited by 101 · see citing works
Quality assessment of randomized control trials in dental research I. Methods
Antczak‐Bouckoms, A., Tang, J., & Chalmers, T. C. (1986) · Journal of Periodontal Research · doi:10.1111/j.1600-0765.1986.tb01464.x · Cited by 95 · see citing works
Maintenance chemotherapy for small cell lung cancer: a critical review of the literature
Sculier, J. P., Berghmans, T., Castaigne, C., Luce, S., Sotiriou, C., Vermylen, P., & Paesmans, M. (1998) · Lung Cancer · doi:10.1016/s0169-5002(97)00084-6 · Cited by 92 · see citing works
Comparison of Carbamazepine and Lithium in the Prophylaxis of Bipolar Disorders
Dardennes, R. M., EVEN, C., Bange, F., & Heim, A. (1995) · The British Journal of Psychiatry · doi:10.1192/bjp.166.3.378 · Cited by 89 · see citing works
A score system for evaluating random control clinical trials of prophylaxis of abdominal surgical wound infection
Evans, M., & Pollock, A. V. (1985) · British journal of surgery · doi:10.1002/bjs.1800720403 · Cited by 85 · see citing works
Quality, Evolution, and Clinical Implications of Randomized, Controlled Trials on the Treatment of Lung Cancer
Nicolucci, A. (1989) · JAMA · doi:10.1001/jama.262.15.2101 · Cited by 78 · see citing works
Haemodilution for acute ischaemic stroke
Asplund, K. (2002) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd000103 · Cited by 76 · see citing works
Position for women during second stage of labour
Gupta, J., & Nikodem, V. (1999) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd002006 · Cited by 74 · see citing works
Large trials vs meta-analysis of smaller trials: how do their results compare?
Cappelleri, J. C. (1996) · JAMA · doi:10.1001/jama.276.16.1332 · Cited by 60 · see citing works
Analysis of randomized controlled trials in laparoscopic surgery
Slim, K., Bousquet, J., Kwiatkowski, F., Pezet, D., & Chipponi, J. (1997) · British journal of surgery · doi:10.1046/j.1365-2168.1997.02753.x · Cited by 55 · see citing works
Methods of Randomized Controlled Clinical Trials in Health Services Research
Balas, E. A., Austin, S., Ewigman, B., BROWN, G. D., & Mitchell, J. A. (1995) · Medical Care · doi:10.1097/00005650-199507000-00005 · Cited by 54 · see citing works
[Evaluation of the methodological quality of randomized therapeutic trials].
Poynard, T. (1988) · PubMed · Cited by 47 · see citing works
A qualitative assessment of randomized controlled trials in otolaryngology
Ah-See, K. W., & Molony, N. (1998) · The Journal of Laryngology & Otology · doi:10.1017/s0022215100140770 · Cited by 45 · see citing works
Spinal Manipulation and Mobilisation for Low Back Pain: A systematic review of randomised clinical trials
Koes, B. W., Assendelft, W., van der Heijden, G., et al. (1996) · Physiotherapy · doi:10.1016/s0031-9406(05)66443-7 · Cited by 43 · see citing works
The Effect of Exercises on Walking Distance of Patients With Intermittent Claudication: A Study of Randomized Clinical Trials
Brandsma, J. W., Robeer, B. G., van den Heuvel, S., et al. (1998) · Physical Therapy · doi:10.1093/ptj/78.3.278 · Cited by 43 · see citing works
Instruments for assessing the quality of drug studies published in the medical literature
Cho, M. K. (1994) · JAMA · doi:10.1001/jama.272.2.101 · Cited by 30 · see citing works
Hyperbaric oxygen for multiple sclerosis
Kleijnen, J., & Knipschild, P. G. (1995) · Acta Neurologica Scandinavica · doi:10.1111/j.1600-0404.1995.tb07017.x · Cited by 29 · see citing works
A cohort study of summary reports of controlled trials
Chalmers, I. (1990) · JAMA · doi:10.1001/jama.263.10.1401 · Cited by 27 · see citing works
The influence of methodologic quality on the conclusion of a landmark meta-analysis on thrombolytic therapy.
Verhagen, A. P., de Vet, H. C. W., Vermeer, F., Widdershoven, J. W. M. G., de Bie, R. A., Kessels, A. G. H., Boers, M., & van den Brandt, P. A. (2002) · PubMed · Cited by 20 · see citing works
Method for Assessment of the Reporting Standard of Clinical Trials with Roentgen Contrast Media
Andrew, E. (1984) · Acta Radiologica Diagnosis · doi:10.1177/028418518402500111 · Cited by 20 · see citing works
An analysis of systematic reviews indicated low incorpororation of results from clinical trial quality assessment
de Craen, A. J. M., van Vliet, H. A. A. M., & Helmerhorst, F. M. (2005) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2004.07.002 · Cited by 18 · see citing works
Critical appraisal of continuing medical education in the rheumatic diseases for primary care physicians
Glazier, R. H., Buchbinder, R., & Bell, M. J. (1995) · Arthritis & Rheumatism · doi:10.1002/art.1780380412 · Cited by 18 · see citing works
The influence of methodologic quality on the conclusion of a landmark meta-analysis on thrombolytic therapy
Doi, S. A. R. (2009) · International Journal of Technology Assessment in Health Care · doi:10.1017/s0266462309091016 · Cited by 15 · see citing works
Influence of Methodology on Outcomes of Randomised Clozapine Trials
Wahlbeck, K., Tuunainen, A., Gilbody, S. M., & Adams, C. E. (2000) · Pharmacopsychiatry · doi:10.1055/s-2000-7971 · Cited by 14 · see citing works
Quality-Adjusted Meta-analysis of the Hypertension/Coronary Dilemma
Holme, I., Ekelund, L. G., Hjermann, I., et al. (1994) · American Journal of Hypertension · doi:10.1093/ajh/7.8.703 · Cited by 14 · see citing works

Flagged first, then by citations

Sorted by citations

The Mixed Methods Appraisal Tool (MMAT) version 2018 for information professionals and researchers
Hong, Q. N., Fàbregues, S., Bartlett, G., Boardman, F., Cargo, M., Dagenais, P., Gagnon, M.-P., Griffiths, F., Nicolau, B., O’Cathain, A., et al. (2018) · Education for Information · doi:10.3233/efi-180221 · Cited by 4,737 · see citing works
“Currently in the literature, it is discouraged to calculate an overall score from the ratings of each criterion (Herbison et al., 2006; Higgins & Green, 2008; Viswanathan et al., 2012).”
Assessment of study quality for systematic reviews: a comparison of the Cochrane Collaboration Risk of Bias Tool and the Effective Public Health Practice Project Quality Assessment Tool: methodological research
Armijo‐Olivo, S., Stiles, C. R., Hagen, N. A., Biondo, P. D., & Cummings, G. G. (2012) · Journal of Evaluation in Clinical Practice · doi:10.1111/j.1365-2753.2010.01516.x · Cited by 1,539 · see citing works
Improving the content validity of the mixed methods appraisal tool: a modified e-Delphi study
Hong, Q. N., Pluye, P., Fàbregues, S., Bartlett, G., Boardman, F., Cargo, M., Dagenais, P., Gagnon, M.-P., Griffiths, F., Nicolau, B., et al. (2019) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2019.03.008 · Cited by 1,082 · see citing works
Scales to Assess the Quality of Randomized Controlled Trials: A Systematic Review
Olivo, S. A., Macedo, L. G., Gadotti, I. C., Fuentes, J., Stanton, T., & Magee, D. J. (2008) · Physical Therapy · doi:10.2522/ptj.20070147 · Cited by 829 · see citing works
The effect of a reduction in alcohol consumption on blood pressure: a systematic review and meta-analysis
Roerecke, M., Kaczorowski, J., Tobe, S. W., Gmel, G., Hasan, O. S. M., & Rehm, J. (2017) · The Lancet Public Health · doi:10.1016/s2468-2667(17)30003-8 · Cited by 552 · see citing works
Camouflaging in autism: A systematic review
Cook, J., Hull, L., Crane, L., & Mandy, W. (2021) · Clinical Psychology Review · doi:10.1016/j.cpr.2021.102080 · Cited by 554 · see citing works
Cognitive–behavioural therapy for the symptoms of schizophrenia: Systematic review and meta-analysis with examination of potential bias
Jauhar, S., McKenna, P. J., Radua, J., Fung, E., Salvador, R., & Laws, K. R. (2014) · British Journal of Psychiatry · doi:10.1192/bjp.bp.112.116285 · Cited by 517 · see citing works
Gaps between knowing and doing: Understanding and assessing the barriers to optimal health care
Cochrane, L. J., Olson, C. A., Murray, S., Dupuis, M., Tooman, T., & Hayes, S. (2007) · Journal of Continuing Education in the Health Professions · doi:10.1002/chp.106 · Cited by 500 · see citing works
Methods to systematically review and meta-analyse observational studies: a systematic scoping review of recommendations
Mueller, M., D’Addario, M., Egger, M., Cevallos, M., Dekkers, O., Mugglin, C., & Scott, P. (2018) · BMC Medical Research Methodology · doi:10.1186/s12874-018-0495-9 · Cited by 400 · see citing works
Association between risk-of-bias assessments and results of randomized trials in Cochrane reviews: The ROBES meta-epidemiologic study
Savović, J., Turner, R. M., Mawdsley, D., Jones, H. E., Beynon, R., Higgins, J. P. T., & Sterne, J. A. C. (2018) · American Journal of Epidemiology · doi:10.1093/aje/kwx344 · Cited by 382 · see citing works
Alcohol consumption and risk of liver cirrhosis: a systematic review and meta-analysis
Roerecke, M., Vafaei, A., Hasan, O. S. M., Chrystoja, B. R., Cruz, M., Lee, R., Neuman, M. G., & Rehm, J. (2019) · American Journal of Gastroenterology · doi:10.14309/ajg.0000000000000340 · Cited by 391 · see citing works
The process writing approach: a meta-analysis
Graham, S., & Sandmel, K. (2011) · The Journal of Educational Research · doi:10.1080/00220671.2010.488703 · Cited by 374 · see citing works
Risk factors for recurrence, complications and mortality in Clostridium difficile infection: a systematic review
Abou Chakra, C. N., Pepin, J., Sirard, S., & Valiquette, L. (2014) · PLoS ONE · doi:10.1371/journal.pone.0098400 · Cited by 328 · see citing works
Alcohol use disorders and mortality: a systematic review and meta-analysis
Roerecke, M., & Rehm, J. (2013) · Addiction · doi:10.1111/add.12231 · Cited by 326 · see citing works
Impact of progressive resistance training on lipids and lipoproteins in adults: A meta-analysis of randomized controlled trials
Kelley, G. A., & Kelley, K. S. (2009) · Preventive Medicine · doi:10.1016/j.ypmed.2008.10.010 · Cited by 263 · see citing works
Comparing Automated Office Blood Pressure Readings With Other Methods of Blood Pressure Measurement for Identifying Patients With Possible Hypertension: A Systematic Review and Meta-analysis
Roerecke, M., Kaczorowski, J., & Myers, M. G. (2019) · JAMA Internal Medicine · doi:10.1001/jamainternmed.2018.6551 · Cited by 216 · see citing works
Cancer incidence in patients with schizophrenia and their first‐degree relatives – a meta‐analysis
Catts, V. S., Catts, S. V., O’Toole, B. I., & Frost, A. D. J. (2008) · Acta Psychiatrica Scandinavica · doi:10.1111/j.1600-0447.2008.01163.x · Cited by 209 · see citing works
Alcohol Consumption as a Risk Factor for Acute and Chronic Pancreatitis: A Systematic Review and a Series of Meta-analyses
Samokhvalov, A. V., Rehm, J., & Roerecke, M. (2015) · EBioMedicine · doi:10.1016/j.ebiom.2015.11.023 · Cited by 200 · see citing works
Limits of meta-analysis: methylphenidate in the treatment of adult attention-deficit hyperactivity disorder
Koesters, M., Becker, T., Kilian, R., Fegert, J., & Weinmann, S. (2009) · Journal of Psychopharmacology · doi:10.1177/0269881108092338 · Cited by 142 · see citing works
Poor Reliability between Cochrane Reviewers and Blinded External Reviewers When Applying the Cochrane Risk of Bias Tool in Physical Therapy Trials
Armijo-Olivo, S., Ospina, M., da Costa, B. R., Egger, M., Saltaji, H., Fuentes, J., Ha, C., & Cummings, G. G. (2014) · PLoS ONE · doi:10.1371/journal.pone.0096920 · Cited by 124 · see citing works
Physical Therapy Interventions for Knee Pain Secondary to Osteoarthritis: A Systematic Review
Wang, S.-Y., Olson-Kellogg, B., Shamliyan, T. A., Choi, J.-Y., Ramakrishnan, R., & Kane, R. L. (2012) · Annals of Internal Medicine · doi:10.7326/0003-4819-157-9-201211060-00007 · Cited by 108 · see citing works
Introduction to systematic reviews and meta‐analysis
McKenzie, J. E., Beller, E. M., & Forbes, A. B. (2016) · Respirology · doi:10.1111/resp.12783 · Cited by 101 · see citing works
Methodological Issues in Systematic Reviews and Meta-Analyses of Observational Studies in Orthopaedic Research
Simunovic, N., Sprague, S., & Bhandari, M. (2009) · Journal of Bone and Joint Surgery · doi:10.2106/jbjs.h.01576 · Cited by 100 · see citing works
Prevalence of Axis-1 psychiatric (with focus on depression and anxiety) disorder and symptomatology among non-medical prescription opioid users in substance use treatment: Systematic review and meta-analyses
Goldner, E. M., Lusted, A., Roerecke, M., Rehm, J., & Fischer, B. (2014) · Addictive Behaviors · doi:10.1016/j.addbeh.2013.11.022 · Cited by 102 · see citing works
Empirical evaluation of which trial characteristics are associated with treatment effect estimates
Dechartres, A., Trinquart, L., Faber, T., & Ravaud, P. (2016) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2016.04.005 · Cited by 85 · see citing works
Assessing risk of bias in human environmental epidemiology studies using three tools: different conclusions from different tools
Eick, S. M., Goin, D. E., Chartres, N., Lam, J., & Woodruff, T. J. (2020) · Systematic Reviews · doi:10.1186/s13643-020-01490-8 · Cited by 82 · see citing works
Inconsistency in the items included in tools used in general health research and physical therapy to evaluate the methodological quality of randomized controlled trials: a descriptive analysis
Armijo-Olivo, S., Fuentes, J., Ospina, M., Saltaji, H., & Hartling, L. (2013) · BMC Medical Research Methodology · doi:10.1186/1471-2288-13-116 · Cited by 60 · see citing works
Estimates of alcohol-related oesophageal cancer burden in Japan: systematic review and meta-analyses
Roerecke, M., Shield, K. D., Higuchi, S., Yoshimura, A., Larsen, E., Rehm, M. X., & Rehm, J. (2015) · Bulletin of the World Health Organization · doi:10.2471/blt.14.142141 · Cited by 44 · see citing works
Methodological quality is underrated in systematic reviews and meta-analyses in health psychology
Oliveras, I., Losilla, J.-M., & Vives, J. (2017) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2017.05.002 · Cited by 15 · see citing works

The effects of chromium supplementation on blood pressure: a systematic review and meta-analysis of randomized clinical trials
Ghanbari, M., Amini, M. R., Djafarian, K., & Shab‐Bidar, S. (2021) · European Journal of Clinical Nutrition · doi:10.1038/s41430-021-00973-8 · Cited by 9 · see citing works
Network Meta-Analysis of the Antihypertensive Effect of Traditional Chinese Exercises on Patients with Essential Hypertension
Li, L., Yang, M., Song, J., et al. (2022) · Journal of Healthcare Engineering · doi:10.1155/2022/9419037 · Cited by 6 · see citing works
MiR-146a rs2910164 polymorphism is associated with hepatocellular carcinoma: a meta-analysis.
Chen, J., Cao, X.-D., & Zhang, H. (2015) · PubMed · Cited by 6 · see citing works
Systematic review and modelling of the cost-effectiveness of cardiac magnetic resonance imaging compared with current existing testing pathways in ischaemic cardiomyopathy
Campbell, F., Thokala, P., Uttley, L., Sutton, A., Sutton, A. J., Al‐Mohammad, A., & Thomas, S. (2014) · Health Technology Assessment · doi:10.3310/hta18590 · Cited by 28 · see citing works
A Comparison of Clinical Follow-Up of Different Total Temporomandibular Joint Replacement Prostheses: A Systematic Review and Meta-Analysis
Zou, L., He, D., & Ellis, E. (2017) · Journal of Oral and Maxillofacial Surgery · doi:10.1016/j.joms.2017.08.022 · Cited by 77 · see citing works

Authors: Herbison, P., Hay-Smith, J., & Gillespie, W. J.
Affiliations: University of Otago (NZ)
Hull York Medical School (GB)
Corresponding author: Herbison, Peter
ORCID: Herbison, Peter
Hay-Smith, Jean
Venue: Journal of Clinical Epidemiology
Publisher: Elsevier BV
Published: 2006-12
Volume: 59
Issue: 12
Pages: 1249.e1-1249.e11
ISSN: 0895-4356 (ISSN-L)
DOI: 10.1016/j.jclinepi.2006.03.008
OpenAlex: W2071955543
Cited by: 207
References: 73
Citations by year: 2026: 3 · 2025: 6 · 2024: 4 · 2023: 5 · 2022: 20 · 2021: 6 · 2020: 6 · 2019: 10 · 2018: 15 · 2017: 14 · 2016: 8 · 2015: 16 · 2014: 17 · 2013: 20 · 2012: 11
FWCI: 2.58
Type: journal article
Language: en
Open access: closed
Licence: www.elsevier.com/tdm/userlicense/1.0/ (tdm)
Locations: Journal of Clinical Epidemiology (publishedVersion)
PubMed (publishedVersion)
Topics: Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
Social Sciences › Sociology and Political Science › Delphi Technique in Research
Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
Keywords: quality scores, trial quality assessment, systematic reviews, binary outcomes, Cochrane Library
MeSH terms: Humans, Reference Standards, Review Literature as Topic, Meta-Analysis as Topic, Reproducibility of Results, Bias, Randomized Controlled Trials as Topic

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Adjustment of meta-analyses on the basis of quality scores should be abandoned

Abstract

Integrity signals

Claims the paper makes

Claims other papers make about this one

Cites

Cited by

Related

Record

About these checks