Methodological quality (risk of bias) assessment tools for primary and secondary medical studies: what are they and which is better?
Open accessAbstract
Methodological quality (risk of bias) assessment is an important step before study initiation usage. Therefore, accurately judging study type is the first priority, and the choosing proper tool is also important. In this review, we introduced methodological quality assessment tools for randomized controlled trial (including individual and cluster), animal study, non-randomized interventional studies (including follow-up study, controlled before-and-after study, before-after/ pre-post study, uncontrolled longitudinal study, interrupted time series study), cohort study, case-control study, cross-sectional study (including analytical and descriptive), observational case series and case reports, comparative effectiveness research, diagnostic study, health economic evaluation, prediction study (including predictor finding study, prediction model impact study, prognostic prediction model study), qualitative study, outcome measurement instruments (including patient - reported outcome measure development, content validity, structural validity, internal consistency, cross-cultural validity/ measurement invariance, reliability, measurement error, criterion validity, hypotheses testing for construct validity, and responsiveness), systematic review and meta-analysis, and clinical practice guideline. The readers of our review can distinguish the types of medical studies and choose appropriate tools. In one word, comprehensively mastering relevant knowledge and implementing more practices are basic requirements for correctly assessing the methodological quality.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Risk of bias was assessed by two independent authors using RoB 2.0, JBI Critical Appraisal Checklists, and the Newcastle-Ottawa Scale.
“Two independent authors used the following tools to assess risk of bias in the included studies: RoB 2.0 for randomized control trials, JBI Critical Appraisal Checklist for Case Series, JBI Critical Appraisal Checklist for Quasi-experimental Studies, and Newcastle–Ottawa Scale for case–control and cohort studies [14,15,16,17].”
From Reconstructive Procedures of the Auricular Concha after Cutaneous Oncologic Surgery: A Systematic Review (2023) · cites this paper as [15]
The sections on randomized trials, non-randomized studies, cohort studies, and case–control studies identify RoB 2.0, the JBI checklist for quasi-experimental studies, and the Newcastle–Ottawa Scale for the stated study designs. The section on case series and case reports does not identify a JBI checklist for case series; it identifies a JBI checklist for case reports instead. The statement that two independent authors performed the assessment describes the citing study’s own procedure, not a finding attributed to this source. (LLM, conf=0.84)
Experimental — accuracy not yet measuredStudy quality was assessed by two independent investigators using the NIH quality assessment tool, with disagreements adjudicated.
“Study quality was assessed using the National Institutes of Health (NIH) quality assessment tool for observational cohort and cross-sectional studies18,19. The quality of each study was evaluated by two independent investigators (MT and RR) and disagreements were resolved by adjudication by the same two reviewers.”
From Prevalence of Depression or Anxiety During Antepartum Hospitalizations for Obstetric Complications (2021) · cites this paper as [18]
The review followed PRISMA 2009 methodology and registered its protocol in PROSPERO.
“We followed PRISMA (Preferred Reporting Items for Systematic review and Meta-Analysis Protocols) 2009 methodology17 for this review and registered the protocol in the PROSPERO International Prospective Register of Systematic Reviews (CRD42020172111).”
From Prevalence of Depression or Anxiety During Antepartum Hospitalizations for Obstetric Complications (2021) · cites this paper as [17]
The cited paper’s Background and Discussion discuss methodological quality and risk-of-bias assessment tools, not whether the citing authors followed PRISMA 2009 or registered their protocol. Searches of our copy for PRISMA, PROSPERO, and the stated registration number found no passage addressing those claims. (LLM, conf=0.76)
Experimental — accuracy not yet measuredWhere I-squared is greater than 50%, the DerSimonian-Laird random-effects model is recommended.
“Where I 2 is greater than 50%, the DerSimonian-Laird randomeffects model was recommended.40”
From Mindfulness-based interventions for psychological wellbeing and quality of life in men with prostate cancer: A systematic review and meta-analysis (2021) · cites this paper as [40]
The cited paper’s section on randomized controlled trials discusses risk-of-bias tools, not how to choose a meta-analysis model. Searches for DerSimonian–Laird, I-squared above 50%, and random-effects models found no passage in our copy addressing the recommendation. (LLM, conf=0.74)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
The Cochrane Collaboration's tool for assessing risk of bias in randomised trials
Higgins, J. P. T., Altman, D. G., Gotzsche, P. C., Juni, P., Moher, D., Oxman, A. D., Savovic, J., Schulz, K. F., Weeks, L., Sterne, J. A. C., et al. (2011) · BMJ · doi:10.1136/bmj.d5928 · Cited by 33,817 · see citing works
RoB 2: a revised tool for assessing risk of bias in randomised trials
Sterne, J. A. C., Savović, J., Page, M. J., Elbers, R. G., Blencowe, N. S., Boutron, I., Cates, C. J., Cheng, H.-Y., Corbett, M. S., Eldridge, S. M., et al. (2019) · BMJ · doi:10.1136/bmj.l4898 · Cited by 33,583 · see citing works
von Elm, E., Altman, D. G., Egger, M., Pocock, S. J., Gøtzsche, P. C., Vandenbroucke, J. P., & Initiative, F. T. S. (2007) · PLoS Medicine · doi:10.1371/journal.pmed.0040296 · Cited by 23,020 · see citing works
von Elm, E., Altman, D. G., Egger, M., Pocock, S. J., Gøtzsche, P. C., & Vandenbroucke, J. P. (2007) · The Lancet · doi:10.1016/s0140-6736(07)61602-x · Cited by 21,320 · see citing works
ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions
Sterne, J. A., Hernán, M. A., Reeves, B. C., Savović, J., Berkman, N. D., Viswanathan, M., Henry, D., Altman, D. G., Ansari, M. T., Boutron, I., et al. (2016) · BMJ · doi:10.1136/bmj.i4919 · Cited by 20,329 · see citing works
(2014) · Medical Entomology and Zoology · Cited by 17,633 · see citing works
Assessing the quality of reports of randomized clinical trials: Is blinding necessary?
Jadad, A. R., Moore, R. A., Carroll, D., Jenkinson, C., Reynolds, D. J. M., Gavaghan, D. J., & McQuay, H. J. (1996) · Controlled Clinical Trials · doi:10.1016/0197-2456(95)00134-4 · Cited by 17,582 · see citing works
Stang, A. (2010) · European Journal of Epidemiology · doi:10.1007/s10654-010-9491-z · Cited by 17,396 · see citing works
Evidence based medicine: what it is and what it isn't
Sackett, D. L., Rosenberg, W. M. C., Gray, J. A. M., Haynes, R. B., & Richardson, W. S. (1996) · BMJ · doi:10.1136/bmj.312.7023.71 · Cited by 15,769 · see citing works
QUADAS-2: a revised tool for the quality assessment of diagnostic accuracy studies
Whiting, P. F., Rutjes, A. W. S., Westwood, M. E., Mallett, S., Deeks, J. J., Reitsma, J. B., Leeflang, M. M. G., Sterne, J. A. C., Bossuyt, P. M. M., & the QUADAS-2 Group* (2011) · Annals of Internal Medicine · doi:10.7326/0003-4819-155-8-201110180-00009 · Cited by 14,199 · see citing works
Shea, B. J., Reeves, B. C., Wells, G., Thuku, M., Hamel, C., Moran, J., Moher, D., Tugwell, P., Welch, V., Kristjansson, E., et al. (2017) · BMJ · doi:10.1136/bmj.j4008 · Cited by 10,636 · see citing works
Downs, S. H., & Black, N. (1998) · Journal of Epidemiology and Community Health · doi:10.1136/jech.52.6.377 · Cited by 8,252 · see citing works
Slim, K., Nini, E., Forestier, D., Kwiatkowski, F., Panis, Y., & Chipponi, J. (2003) · ANZ Journal of Surgery · doi:10.1046/j.1445-2197.2003.02748.x · Cited by 7,603 · see citing works
Schulz (1995) · doi:10.1001/jama.1995.03520290060030 · Cited by 5,634 · see citing works
Reliability of the PEDro Scale for Rating Quality of Randomized Controlled Trials
Maher, C. G., Sherrington, C., Herbert, R. D., Moseley, A. M., & Elkins, M. (2003) · Physical Therapy · doi:10.1093/ptj/83.8.713 · Cited by 4,808 · see citing works
Guyatt, G. H. (1992) · JAMA · doi:10.1001/jama.1992.03490170092032 · Cited by 4,447 · see citing works
SYRCLE’s risk of bias tool for animal studies
Hooijmans, C. R., Rovers, M. M., de Vries, R. B., Leenaars, M., Ritskes-Hoitinga, M., & Langendam, M. W. (2014) · BMC Medical Research Methodology · doi:10.1186/1471-2288-14-43 · Cited by 4,415 · see citing works
Development of AMSTAR: A measurement tool to assess the methodological quality of systematic reviews
Shea, B. J., Grimshaw, J. M., Wells, G. A., Boers, M., Andersson, N., Hamel, C., Porter, A. C., Tugwell, P., Moher, D., & Bouter, L. M. (2007) · BMC Medical Research Methodology · doi:10.1186/1471-2288-7-10 · Cited by 4,374 · see citing works
Yeh, M. (2014) · Journal of Nursing Research · doi:10.1097/jnr.0000000000000065 · Cited by 3,836 · see citing works
AGREE II: advancing guideline development, reporting and evaluation in health care
Brouwers, M. C., Kho, M. E., Browman, G. P., Burgers, J. S., Cluzeau, F., Feder, G., Fervers, B., Graham, I. D., Grimshaw, J., Hanna, S. E., et al. (2010) · Canadian Medical Association Journal · doi:10.1503/cmaj.090449 · Cited by 3,790 · see citing works
Whiting, P., Rutjes, A. W., Reitsma, J. B., Bossuyt, P. M., & Kleijnen, J. (2003) · BMC Medical Research Methodology · doi:10.1186/1471-2288-3-25 · Cited by 3,592 · see citing works
COSMIN guideline for systematic reviews of patient-reported outcome measures
Prinsen, C. A. C., Mokkink, L. B., Bouter, L. M., Alonso, J., Patrick, D. L., de Vet, H. C. W., & Terwee, C. B. (2018) · Quality of Life Research · doi:10.1007/s11136-018-1798-3 · Cited by 3,345 · see citing works
Assessing the quality of controlled clinical trials
Jüni, P., Altman, D. G., & Egger, M. (2001) · BMJ · doi:10.1136/bmj.323.7303.42 · Cited by 3,309 · cited 1× · see citing works
“There are three types of tools: scales, checklists, and items [10, 11].”
MethodAssessing Bias in Studies of Prognostic Factors
Hayden, J. A., van der Windt, D. A., Cartwright, J. L., Côté, P., & Bombardier, C. (2013) · Annals of Internal Medicine · doi:10.7326/0003-4819-158-4-201302190-00009 · Cited by 3,287 · see citing works
Munn, Z., Moola, S., Lisy, K., Riitano, D., & Tufanaru, C. (2015) · International Journal of Evidence-Based Healthcare · doi:10.1097/xeb.0000000000000054 · Cited by 3,131 · see citing works
PROBAST: A Tool to Assess the Risk of Bias and Applicability of Prediction Model Studies
Wolff, R. F., Moons, K. G. M., Riley, R. D., Whiting, P. F., Westwood, M., Collins, G. S., Reitsma, J. B., Kleijnen, J., Mallett, S., & for the PROBAST Group† (2019) · Annals of Internal Medicine · doi:10.7326/m18-1376 · Cited by 2,984 · see citing works
Evaluating non-randomised intervention studies
Deeks, J., Dinnes, J., D'Amico, R., Sowden, A., Sakarovitch, C., Song, F., Petticrew, M., & Altman, D. (2003) · Health Technology Assessment · doi:10.3310/hta7270 · Cited by 2,907 · see citing works
Moher, D., Schulz, K. F., & Altman, D. G. (2001) · BMC Medical Research Methodology · doi:10.1186/1471-2288-1-2 · Cited by 2,755 · see citing works
Essentials of Nursing Research: Appraising Evidence for Nursing Practice
Polit, D. F., & Beck, C. T. (2013) · Cited by 2,621 · see citing works
Critical appraisal skills programme
Singh, J. (2013) · Journal of Pharmacology and Pharmacotherapeutics · doi:10.4103/0976-500x.107697 · Cited by 2,571 · see citing works
Moher, D. (2001) · JAMA · doi:10.1001/jama.285.15.1987 · Cited by 2,545 · see citing works
COSMIN Risk of Bias checklist for systematic reviews of Patient-Reported Outcome Measures
Mokkink, L. B., de Vet, H. C. W., Prinsen, C. A. C., Patrick, D. L., Alonso, J., Bouter, L. M., & Terwee, C. B. (2018) · Quality of Life Research · doi:10.1007/s11136-017-1765-4 · Cited by 2,479 · see citing works
Verhagen, A., de Vet, H. D., de Bie, R. D., Kessels, A., Boers, M., Bouter, L., & Knipschild, P. (1998) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(98)00131-0 · Cited by 2,473 · see citing works
Development of a critical appraisal tool to assess the quality of cross-sectional studies (AXIS)
Downes, M. J., Brennan, M. L., Williams, H. C., & Dean, R. S. (2016) · BMJ Open · doi:10.1136/bmjopen-2016-011458 · Cited by 2,376 · see citing works
ROBIS: A new tool to assess risk of bias in systematic reviews was developed
Whiting, P., Savović, J., Higgins, J. P. T., Caldwell, D. M., Reeves, B. C., Shea, B., Davies, P., Kleijnen, J., & Churchill, R. (2016) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2015.06.005 · Cited by 2,272 · see citing works
Zeng, X., Zhang, Y., Kwong, J. S. W., Zhang, C., Li, S., Sun, F., Niu, Y., & Du, L. (2015) · Journal of Evidence-Based Medicine · doi:10.1111/jebm.12141 · Cited by 2,082 · see citing works
Self-citationGuidelines for authors and peer reviewers of economic submissions to the BMJ
Drummond, M. F., & Jefferson, T. O. (1996) · BMJ · doi:10.1136/bmj.313.7052.275 · Cited by 1,827 · see citing works
Going from evidence to recommendations
Guyatt, G. H., Oxman, A. D., Kunz, R., Falck-Ytter, Y., Vist, G. E., Liberati, A., & Schünemann, H. J. (2008) · BMJ · doi:10.1136/bmj.39493.646875.ae · Cited by 1,810 · see citing works
Consolidated Health Economic Evaluation Reporting Standards (CHEERS) statement
Husereau, D., Drummond, M., Petrou, S., Carswell, C., Moher, D., Greenberg, D., Augustovski, F., Briggs, A. H., Mauskopf, J., Loder, E., et al. (2013) · BMJ · doi:10.1136/bmj.f1049 · Cited by 1,776 · see citing works
Sanderson, S., Tatt, I. D., & Higgins, J. P. (2007) · International Journal of Epidemiology · doi:10.1093/ije/dym018 · Cited by 1,531 · see citing works
(2003) · Quality and Safety in Health Care · doi:10.1136/qhc.12.1.18 · Cited by 1,446 · see citing works
Andrews, J., Guyatt, G., Oxman, A. D., Alderson, P., Dahm, P., Falck-Ytter, Y., Nasser, M., Meerpohl, J., Post, P. N., Kunz, R., et al. (2013) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2012.03.013 · Cited by 1,416 · see citing works
Update of the Stroke Therapy Academic Industry Roundtable Preclinical Recommendations
Fisher, M., Feuerstein, G., Howells, D. W., Hurn, P. D., Kent, T. A., Savitz, S. I., & Lo, E. H. (2009) · Stroke · doi:10.1161/strokeaha.108.541128 · Cited by 1,352 · see citing works
A method for assessing the quality of a randomized control trial
Chalmers, T. C., Smith, H., Blackburn, B., Silverman, B., Schroeder, B., Reitman, D., & Ambroz, A. (1981) · Controlled Clinical Trials · doi:10.1016/0197-2456(81)90056-8 · Cited by 1,349 · see citing works
Roundtable, S. T. A. I. (1999) · Stroke · doi:10.1161/01.str.30.12.2752 · Cited by 1,258 · see citing works
Meta-Analyses of Randomized Controlled Trials
Sacks, H. S., Berrier, J., Reitman, D., Ancona-Berk, V. A., & Chalmers, T. C. (1987) · New England Journal of Medicine · doi:10.1056/nejm198702193160806 · Cited by 1,253 · see citing works
(1997) · Europe PMC (PubMed Central) · Cited by 1,226 · see citing works
Factors relevant to the validity of experiments in social settings.
Campbell, D. T. (1957) · Psychological Bulletin · doi:10.1037/h0040950 · Cited by 1,138 · see citing works
Mokkink, L. B., Prinsen, C. A. C., Bouter, L. M., de Vet, H. C. W., & Terwee, C. B. (2016) · Brazilian Journal of Physical Therapy · doi:10.1590/bjpt-rbf.2014.0143 · Cited by 915 · see citing works
(2003) · Digital Education Resource Archive (University College London) · Cited by 902 · see citing works
Systems to rate the strength of scientific evidence.
West, S. L., King, V., Carey, T. S., Lohr, K. N., Mckoy, N. A., Sutton, S. F., & Lux, L. J. (2002) · PubMed Central · Cited by 873 · see citing works
The Medical Review Article: State of the Science
MULROW, C. D. (1987) · Annals of Internal Medicine · doi:10.7326/0003-4819-106-3-485 · Cited by 856 · see citing works
Comparison of Treatment Effects between Animal Experiments and Clinical Trials: Systematic Review
Perel, P., Roberts, I., Sena, E., Wheble, P., Briscoe, C., Sandercock, P., Macleod, M., Mignini, L. E., Jayaram, P., & Khan, K. S. (2007) · BMJ · doi:10.1136/bmj.39048.407928.be · Cited by 827 · see citing works
The CARE guidelines: consensus-based clinical case report guideline development
Gagnier, J. J., Kienle, G. S., Altman, D. G., Moher, D., Sox, H. C., & Riley, D. S. (2013) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2013.08.003 · Cited by 823 · see citing works
Higgins, J. (2011) · Medical Entomology and Zoology · Cited by 816 · see citing works
Validation of an index of the quality of review articles
Oxman, A. D., & Guyatt, G. H. (1991) · Journal of Clinical Epidemiology · doi:10.1016/0895-4356(91)90160-b · Cited by 788 · see citing works
Randomised trials in surgery: problems and possible solutions
McCulloch, P. G., Taylor, I., Sasako, M., Lovett, B., & Griffin, D. (2002) · BMJ · doi:10.1136/bmj.324.7351.1448 · Cited by 761 · see citing works
An overview of clinical research: the lay of the land
Grimes, D. A., & Schulz, K. F. (2002) · The Lancet · doi:10.1016/s0140-6736(02)07283-5 · Cited by 734 · see citing works
Pooling of Animal Experimental Data Reveals Influence of Study Design and Publication Bias
Macleod, M., O'Collins, T., Howells, D., & Donnan, G. (2004) · Stroke · doi:10.1161/01.str.0000125719.25853.20 · Cited by 692 · see citing works
Consolidated health economic evaluation reporting standards (CHEERS) statement
Husereau, D., Drummond, M., Petrou, S., Carswell, C., Moher, D., Greenberg, D., Augustovski, F., Briggs, A. H., Mauskopf, J., & Loder, E. (2013) · Value in Health · doi:10.1016/j.jval.2013.02.010 · Cited by 657 · see citing works
Summing up evidence: one answer is not always enough
Lau, J., Ioannidis, J. P. A., & Schmid, C. H. (1998) · The Lancet · doi:10.1016/s0140-6736(97)08468-7 · Cited by 646 · see citing works
Reporting and Methods in Clinical Prediction Research: A Systematic Review
Bouwmeester, W., Zuithoff, N. P. A., Mallett, S., Geerlings, M. I., Vergouwe, Y., Steyerberg, E. W., Altman, D. G., & Moons, K. G. M. (2012) · PLoS Medicine · doi:10.1371/journal.pmed.1001221 · Cited by 600 · see citing works
Cohort studies: marching towards outcomes
Grimes, D. A., & Schulz, K. F. (2002) · The Lancet · doi:10.1016/s0140-6736(02)07500-1 · Cited by 491 · see citing works
Systematic Reviews: Checklists for review articles
Oxman, A. D. (1994) · BMJ · doi:10.1136/bmj.309.6955.648 · Cited by 480 · see citing works
Browman, G. P., Levine, M. N., Mohide, E. A., Hayward, R. S., Pritchard, K. I., Gafni, A., & Laupacis, A. (1995) · Journal of Clinical Oncology · doi:10.1200/jco.1995.13.2.502 · Cited by 441 · see citing works
Development and validation of methods for assessing the quality of diagnostic accuracy studies
Whiting, P. F., Rutjes, A. W., Dinnes, J., Reitsma, J. B., Bossuyt, P. M., & Kleijnen, J. (2004) · Health Technology Assessment · doi:10.3310/hta8250 · Cited by 372 · see citing works
Risk of bias versus quality assessment of randomised controlled trials: cross sectional study
Hartling, L., Ospina, M., Liang, Y., Dryden, D. M., Hooton, N., Krebs Seida, J., & Klassen, T. P. (2009) · BMJ · doi:10.1136/bmj.b4012 · Cited by 348 · see citing works
Vlayen, J., Aertgeerts, B., Hannes, K., Sermeus, W., & Ramaekers, D. (2005) · International Journal for Quality in Health Care · doi:10.1093/intqhc/mzi027 · Cited by 313 · see citing works
Users' Guides to the Medical Literature
Drummond, M. (1997) · JAMA · doi:10.1001/jama.1997.03540430064035 · Cited by 303 · see citing works
Hannes, K., & Lockwood, C. S. (2011) · Journal of Advanced Nursing · doi:10.1111/j.1365-2648.2011.05636.x · Cited by 287 · see citing works
A systematic review classifies sources of bias and variation in diagnostic test accuracy studies
Whiting, P. F., Rutjes, A. W. S., Westwood, M. E., & Mallett, S. (2013) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2013.05.014 · Cited by 283 · see citing works
Integrating evidence into clinical practice: an alternative to evidence‐based approaches
Tonelli, M. R. (2006) · Journal of Evaluation in Clinical Practice · doi:10.1111/j.1365-2753.2004.00551.x · Cited by 252 · see citing works
Assessing measurement in health: Beyond reliability and validity
Polit, D. F. (2015) · International Journal of Nursing Studies · doi:10.1016/j.ijnurstu.2015.07.002 · Cited by 210 · see citing works
Wong, S. S.-L., Wilczynski, N. L., & Haynes, R. B. (2004) · PubMed · Cited by 203 · see citing works
Progress in using systematic reviews of animal studies to improve translational research
Hooijmans, C. R., & Ritskes-Hoitinga, M. (2013) · PLoS Medicine · doi:10.1371/journal.pmed.1001482 · Cited by 195 · see citing works
A systematic review of the effectiveness of critical appraisal skills training for clinicians
Taylor, R. S., Reeves, B. C., Ewings, P., Binns, S. H., Keast, J., & Mears, R. (2000) · Medical Education · doi:10.1046/j.1365-2923.2000.00574.x · Cited by 182 · see citing works
Treatment of Pulmonary Tuberculosis with Streptomycin and Para-Amino-Salicylic Acid
(1950) · BMJ · doi:10.1136/bmj.2.4688.1073 · Cited by 175 · see citing works
Manchikanti, L. (2008) · Pain Physician · doi:10.36076/ppj.2008/11/161 · Cited by 157 · see citing works
GRACE principles: recognizing high-quality observational studies of comparative effectiveness.
Dreyer, N. A., Schneeweiß, S., McNeil, B. J., Berger, M. L., Walker, A. M., Ollendorf, D. A., & Gliklich, R. E. (2010) · PubMed · Cited by 156 · see citing works
How to Practice Evidence-Based Medicine
Swanson, J., Schmitz, D., & Chung, K. C. (2010) · Plastic & Reconstructive Surgery · doi:10.1097/prs.0b013e3181dc54ee · Cited by 138 · see citing works
Aid to the Evaluation of Therapeutic Studies
REISCH, J. S., Tyson, J. E., & Mize, S. G. (1989) · PEDIATRICS · doi:10.1542/peds.84.5.815 · Cited by 123 · see citing works
The Global Rating Scale complements the AGREE II in advancing the quality of practice guidelines
Brouwers, M. C., Kho, M. E., Browman, G. P., Burgers, J. S., Cluzeau, F., Feder, G. S., Fervers, B., Graham, I. D., Grimshaw, J., Hanna, S., et al. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2011.10.008 · Cited by 117 · see citing works
Oxman, A. D., Schünemann, H. J., & Fretheim, A. (2006) · Health Research Policy and Systems · doi:10.1186/1478-4505-4-20 · Cited by 116 · see citing works
Schueler, S., Schuetz, G. M., & Dewey, M. (2012) · Annals of Internal Medicine · doi:10.7326/0003-4819-156-4-201202210-00018 · Cited by 91 · see citing works
Fisher, Bradford Hill, and randomization.
Armitage, P. (2003) · International Journal of Epidemiology · doi:10.1093/ije/dyg286 · Cited by 86 · see citing works
Doctors’ Perceptions and Use of Evidence-Based Medicine
Swennen, M. H. J., van der Heijden, G. J. M. G., Boeije, H., van Rheenen, N., Verheul, F., van der Graaf, Y., & Kalkman, C. J. (2013) · Academic Medicine · doi:10.1097/acm.0b013e31829ed3cc · Cited by 80 · see citing works
Chapman, P. F. (1934) · BMJ · doi:10.1136/bmj.2.3857.1073-b · Cited by 78 · see citing works
Levin, A. R. (2001) · Annals of Internal Medicine · doi:10.7326/0003-4819-135-4-200108210-00035 · Cited by 74 · see citing works
Evidence-Based Medicine and Practice Guidelines: An Overview
Woolf, S. H. (2000) · Cancer Control · doi:10.1177/107327480000700411 · Cited by 70 · see citing works
Reproducibility of the Portuguese version of the PEDro Scale
Shiwa, S. R., Costa, L. O. P., Costa, L. D. C. M., Moseley, A. M., Hespanhol, L., Venancio, R. C., Ruggero, C. R., Sato, T. D. O., & Lópes, A. D. (2011) · Cadernos de Saúde Pública · doi:10.1590/s0102-311x2011001000019 · Cited by 68 · see citing works
Archibald Cochrane (1909-1988): the father of evidence-based medicine
Stavrou, A., Challoumas, D., & Dimitrakakis, G. (2013) · Interactive Cardiovascular and Thoracic Surgery · doi:10.1093/icvts/ivt451 · Cited by 61 · see citing works
Joanna Briggs Institute: An Evidence-Based Practice Database
Vardell, E., & Malloy, M. (2013) · Medical Reference Services Quarterly · doi:10.1080/02763869.2013.837734 · Cited by 60 · see citing works
Evidence-based health economics: from effectiveness to efficiency in systematic review.
Thoonen, B., & Schermer, T. (2003) · Family Practice · doi:10.1093/oxfordjournals.fampra.a001510 · Cited by 56 · see citing works
Quality assessment of clinical guidelines in China: 1993 - 2010.
Chen, Y., Yao, L., Xiao, X.-J., Wang, Q., Wang, Z.-H., Liang, F., Liang, H., Wang, X., Shen, X., Xie, C.-C., et al. (2012) · PubMed · doi:10.3760/cma.j.issn.0366-6999.2012.20.011 · Cited by 50 · see citing works
Evaluation of a programme of workshops for promoting the teaching of critical appraisal skills
Ibbotson, Grimshaw, J., & Grant (1998) · Medical Education · doi:10.1046/j.1365-2923.1998.00256.x · Cited by 46 · see citing works
Willis, B. H., & Quigley, M. (2011) · BMC Medical Research Methodology · doi:10.1186/1471-2288-11-27 · Cited by 43 · see citing works
Strom, B. L. 1. (2000) · Cited by 43 · see citing works
Evidence Based Health Economics: From Effectiveness to Efficiency in Systematic Review
Donaldson, C., Mugford, M., & Vale, L. D. (2002) · doi:10.25405/ncl.34159329 · Cited by 40 · see citing works
Periodontal disease and risk of benign prostate hyperplasia: a cross-sectional study
Wu, L., Li, B., Wang, X., Wang, C.-Y., Zi, H., Weng, H., Huang, Q., Zhu, Y.-J., & Zeng, X. (2019) · Military Medical Research · doi:10.1186/s40779-019-0223-8 · Cited by 39 · see citing works
Self-citationPractical Guide to Understanding the Need for Clinical Practice Guidelines
Neely, J. G., Graboyes, E. M., Paniello, R. C., Sequeira, S. M., & Grindler, D. J. (2013) · Otolaryngology · doi:10.1177/0194599813487501 · Cited by 38 · see citing works
Commentary on the role of expert opinion in developing evidence-based guidelines
Eibling, D. E., Fried, M. P., Blitzer, A., & Postma, G. (2013) · The Laryngoscope · doi:10.1002/lary.24175 · Cited by 35 · see citing works
An Introduction to Economic Evaluation: What's in a Name?
Hoch, J. S., & Dewa, C. S. (2005) · The Canadian Journal of Psychiatry · doi:10.1177/070674370505000305 · Cited by 30 · see citing works
From bench-top to chair-side: How scientific evidence is incorporated into clinical practice
Tracy, S. L. (2013) · Dental Materials · doi:10.1016/j.dental.2013.08.200 · Cited by 26 · see citing works
Zhang, J., Wang, Y., Weng, H., et al. (2019) · BMC Cancer · doi:10.1186/s12885-019-6304-y · Cited by 24 · see citing works
Self-citationThe quality of clinical practice guidelines in C hina: a systematic assessment
Hu, J., Chen, R., Wu, S., Tang, J.-L., Leng, G. C., Kunnamo, I., Yang, Z., Wang, W., Hua, X., Zhang, Y., et al. (2012) · Journal of Evaluation in Clinical Practice · doi:10.1111/j.1365-2753.2012.01893.x · Cited by 22 · see citing works
Self-citationMeta-analyses, multivariate analyses, and coping with the play of chance
Clarke, M. J., & Chalmers, I. (1998) · The Lancet · doi:10.1016/s0140-6736(05)79032-2 · Cited by 22 · see citing works
An alternative pathway for preclinical research in fluid management
Sibbald, W. J. (2000) · Critical Care · doi:10.1186/cc970 · Cited by 22 · see citing works
Systematic Reviews A Logical Methodological Extension of Evidence‐based Medicine
Gallagher, E. J. (1999) · Academic Emergency Medicine · doi:10.1111/j.1553-2712.1999.tb00142.x · Cited by 15 · see citing works
Henig, O., Yahav, D., Leibovici, L., & Paul, M. (2013) · Clinical Microbiology and Infection · doi:10.1111/1469-0691.12348 · Cited by 14 · see citing works
Gold, C., Erkkilä, J., & Crawford, M. (2012) · Acta Psychiatrica Scandinavica · doi:10.1111/j.1600-0447.2012.01922.x · Cited by 13 · see citing works
Li, B., Yu, Z., Wang, C., et al. (2019) · Frontiers in Pharmacology · doi:10.3389/fphar.2019.01426 · Cited by 13 · see citing works
Evidence-based medicine – are we boiling the frog?
Muckart, D. J. J. (2013) · South African Medical Journal · doi:10.7196/samj.6805 · Cited by 11 · see citing works
Using preappraised evidence sources to guide practice decisions
Chapa, D. W., Hartung, M. K., Mayberry, L. J., & Pintz, C. (2012) · Journal of the American Association of Nurse Practitioners · doi:10.1111/j.1745-7599.2012.00787.x · Cited by 7 · see citing works
Evidence-based medicine should be based on science
Tunguy-Desmarais, G. P., & Muckart, D. J. J. (2013) · South African Medical Journal · doi:10.7196/samj.7406 · Cited by 4 · see citing works
Flagged first, then by citations
Research works printed but not linked: 1
- [49]Crombie I Pocket guide to critical appraisal: Oxford 1996 UK John Wiley & Sons, Ltd
Cited by
Sorted by citations
Hertenstein, E., Trinca, E., Wunderlin, M., Schneider, C. L., Züst, M. A., Fehér, K. D., Su, T., Straten, A. V., Berger, T., Baglioni, C., et al. (2022) · Sleep Medicine Reviews · doi:10.1016/j.smrv.2022.101597 · Cited by 391 · see citing works
Njagi, P., Groot, W., Arsenijevic, J., Dyer, S., Mburu, G., & Kiarie, J. (2023) · Human Reproduction Open · doi:10.1093/hropen/hoad007 · Cited by 277 · see citing works
Drug checking services for people who use drugs: a systematic review
Maghsoudi, N., Tanguay, J., Scarfone, K., Rammohan, I., Ziegler, C., Werb, D., & Scheim, A. I. (2022) · Addiction · doi:10.1111/add.15734 · Cited by 261 · see citing works
Chiu, K., Clark, D. M., & Leigh, E. (2021) · Journal of Affective Disorders · doi:10.1016/j.jad.2020.10.055 · Cited by 155 · see citing works
Alvandi, E., Wong, W. K. M., Joglekar, M. V., Spring, K. J., & Hardikar, A. A. (2022) · BMC Medicine · doi:10.1186/s12916-022-02529-4 · Cited by 101 · see citing works
Drukker, M., Weltens, I., van Hooijdonk, C. F. M., Vandenberk, E., & Bak, M. (2021) · Frontiers in Research Metrics and Analytics · doi:10.3389/frma.2021.675071 · Cited by 75 · see citing works
Park, H. Y., Suh, C. H., Woo, S., Kim, P. H., & Kim, K. W. (2022) · Korean Journal of Radiology · doi:10.3348/kjr.2021.0808 · Cited by 61 · see citing works
Prevalence of Depression or Anxiety During Antepartum Hospitalizations for Obstetric Complications
Toscano, M., Royzer, R., Castillo, D., Li, D., & Poleshuck, E. (2021) · Obstetrics & Gynecology · doi:10.1097/aog.0000000000004335 · Cited by 55 · see citing works
Gosling, H., Pratt, D., Montgomery, H., & Lea, J. (2022) · Journal of Affective Disorders · doi:10.1016/j.jad.2021.12.091 · Cited by 51 · see citing works
Chen, J. Y., Fernandez, K., Fadadu, R. P., Reddy, R., Kim, M.-O., Tan, J., & Wei, M. L. (2025) · JAMA Dermatology · doi:10.1001/jamadermatol.2024.4382 · Cited by 43 · see citing works
Jahangiry, L., Aune, D., & Farhangi, M. A. (2022) · Nutrition, Metabolism and Cardiovascular Diseases · doi:10.1016/j.numecd.2022.08.004 · Cited by 36 · see citing works
Moslehi, N., Zeraattalab-Motlagh, S., Rahimi Sakak, F., Shab-Bidar, S., Tehrani, F. R., & Mirmiran, P. (2023) · Nutrition Reviews · doi:10.1093/nutrit/nuac075 · Cited by 28 · see citing works
Le, G. H., Wong, S., Badulescu, S., Au, H., Di Vincenzo, J. D., Gill, H., Phan, L., Rhee, T. G., Ho, R., Teopiz, K. M., et al. (2024) · Journal of Affective Disorders · doi:10.1016/j.jad.2024.03.165 · Cited by 30 · see citing works
Nnate, D. A., Anyachukwu, C. C., Igwe, S. E., & Abaraogu, U. O. (2021) · Psycho-Oncology · doi:10.1002/pon.5749 · Cited by 24 · see citing works
A meta-analysis of teacher well-being: A job demands and resources perspective
Li, Y., Chen, J., Lu, J., & Wang, X. (2025) · Educational Research Review · doi:10.1016/j.edurev.2025.100719 · Cited by 19 · see citing works
Self-citationUlugut, H., Trieu, C., Groot, C., van ’t Hooft, J. J., Tijms, B. M., Scheltens, P., Ossenkoppele, R., Barkhof, F., van den Heuvel, O. A., & Pijnenburg, Y. A. L. (2023) · Biological Psychiatry · doi:10.1016/j.biopsych.2022.05.028 · Cited by 16 · see citing works
Neuroanatomical regions associated with non-progressive dysarthria post-stroke: a systematic review
Summaka, M., Hannoun, S., Harati, H., Daoud, R., Zein, H., Estephan, E., Naim, I., & Nasser, Z. (2022) · BMC Neurology · doi:10.1186/s12883-022-02877-x · Cited by 16 · see citing works
Borinuoluwa, R., & Ahmed, Z. (2023) · Trauma Care · doi:10.3390/traumacare3010002 · Cited by 13 · see citing works
Shete-Motgi, A., Shete, M., Bhate, K., Kshirsagar, K., & Kakodkar, P. (2022) · Journal of Indian Academy of Oral Medicine and Radiology · doi:10.4103/jiaomr.jiaomr_273_21 · Cited by 7 · see citing works
Moreno-Vazquez, S., Antoñanzas, J., Oteiza-Rius, I., Redondo, P., & Salido-Vallejo, R. (2023) · Journal of Clinical Medicine · doi:10.3390/jcm12206521 · Cited by 6 · see citing works
Bahji, A., Danilewitz, M., Vazquez, G., & Patten, S. (2021) · Canadian Journal of Addiction · doi:10.1097/cxa.0000000000000123 · Cited by 6 · see citing works
Jungersen, R. C., Baccaro, V. K., Hvid, L. G., Dalgas, U., & Hoegh, M. (2026) · European Journal of Pain · doi:10.1002/ejp.70375 · Cited by 1 · see citing works
Gastrointestinal symptoms and disorders in multiple sclerosis: a systematic review and meta-analysis
Sredanović, M., Hentzen, C., Adamec, I., Chesnel, C., Krbot Skorić Krbot Skorić, M., & Habek, M. (2026) · Journal of Neurology · doi:10.1007/s00415-026-14041-3 · Cited by 1 · see citing works
- Authors
- Ma, L.-L., Wang, Y.-Y., Yang, Z.-H., Huang, D., Weng, H., & Zeng, X.-T.
- Affiliations
- Wuhan University (CN)
- Corresponding author
- Zeng, Xian-Tao
- Venue
- Military Medical Research
- Publisher
- Elsevier BV
- Published
- 2020-02-29
- Volume
- 7
- Issue
- 1
- Pages
- 7-7
- ISSN
- 2054-9369 (ISSN-L)
- PMC
- PMC7049186
- OpenAlex
- W3011962481
- Cited by
- 1,648
- References
- 115
- Citations by year
- 2026: 132 · 2025: 280 · 2024: 374 · 2023: 353 · 2022: 317 · 2021: 160 · 2020: 30 · 2019: 1 · 2012: 1
- FWCI
- 44.29
- Type
- journal article
- Language
- en
- Open access
- diamond
- doi.org
- Locations
- Military Medical Research (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- Entrusted Project of National commission on health and health of China (No. [2019]099)
- National Key Research and Development Plan of China (2016YFC0106300)
- Nature Science Foundation of Hubei Province (2019FFB03902)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Health Sciences › Health Professions › Health Information Management › Methodologies in Health Research and Practice
- Keywords
- methodological quality assessment, risk of bias, quality assessment tools, randomized controlled trials, observational studies, cohort studies, case-control study, cross-sectional study, diagnostic studies, health economic evaluation, prediction model, patient-reported outcome measures, measurement validity, reliability statistics, systematic reviews, comparative effectiveness research, animal studies
- MeSH terms
- Animals, Humans, Psychometrics, Research, Research Design, Reproducibility of Results, Bias
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
