How to critically appraise an article
The DOI matches its Crossref record; 64 of the 67 references it cites were found in a registry or an index.
Abstract
Critical appraisal is a systematic process used to identify the strengths and weaknesses of a research article in order to assess the usefulness and validity of research findings. The most important components of a critical appraisal are an evaluation of the appropriateness of the study design for the research question and a careful assessment of the key methodological features of this design. Other factors that also should be considered include the suitability of the statistical methods used and their subsequent interpretation, potential conflicts of interest and the relevance of the research to one's own practice. This Review presents a 10-step guide to critical appraisal that aims to assist clinicians to identify the most relevant high-quality studies available to guide their clinical practice.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
Meta-analysis of Observational Studies in Epidemiology<SUBTITLE>A Proposal for Reporting</SUBTITLE>
Stroup, D. F. (2000) · JAMA · doi:10.1001/jama.283.15.2008 · Cited by 20,623
Elm, E. V., Altman, D. G., Egger, M., Pocock, S. J., Gøtzsche, P. C., & Vandenbroucke, J. P. (2007) · BMJ · doi:10.1136/bmj.39335.541782.ad · Cited by 10,766
Evidence-Based Medicine: How to Practice and Teach EBM
Sackett, D. L. (2000) · Cited by 5,950
Schulz (1995) · doi:10.1001/jama.1995.03520290060030 · Cited by 5,634
Development of AMSTAR: A measurement tool to assess the methodological quality of systematic reviews
Shea, B. J., Grimshaw, J. M., Wells, G. A., Boers, M., Andersson, N., Hamel, C., Porter, A. C., Tugwell, P., Moher, D., & Bouter, L. M. (2007) · BMC Medical Research Methodology · doi:10.1186/1471-2288-7-10 · Cited by 4,374
Qualitative research in health care: Assessing quality in qualitative research
Mays, N. (2000) · BMJ · doi:10.1136/bmj.320.7226.50 · Cited by 3,742
Publication bias in clinical research
Easterbrook, P. J., Gopalan, R., Berlin, J. A., & Matthews, D. R. (1991) · The Lancet · doi:10.1016/0140-6736(91)90201-y · Cited by 3,109
Moher, D., Schulz, K. F., & Altman, D. G. (2001) · BMC Medical Research Methodology · doi:10.1186/1471-2288-1-2 · Cited by 2,755
Piaggio, G., Elbourne, D. R., Pocock, S. J., Evans, S. J. W., Altman, D. G., & CONSORT Group, F. T. (2012) · JAMA · doi:10.1001/jama.2012.87802 · Cited by 2,428
Porta, M. (2026) · doi:10.1093/acref/9780197663639.001.0001 · Cited by 2,175
Guide to Clinical Preventive Services
Wesley Allen, T. (1991) · The Journal of the American Osteopathic Association · doi:10.1515/jom-1991-900515 · Cited by 2,129
Evidence-Based Medicine: How to Practice and Teach EBM
Satya-Murti, S. (2000) · JAMA: The Journal of the American Medical Association · doi:10.1001/jama.284.18.2382 · Cited by 1,643
Host Range and Emerging and Reemerging Pathogens
Woolhouse, M. E. J., & Gowtage-Sequeria, S. (2005) · Emerging Infectious Diseases · doi:10.3201/eid1112.050997 · Cited by 1,586
The Cochrane Database of Systematic Reviews
Handoll, H. H. (2004) · The British Journal of Hand Therapy · doi:10.1177/175899830400900201 · Cited by 1,513
Users' guides to the medical literature : a manual for evidence-based clinical practice
Guyatt, G. H., Rennie, D., Meade, M. O., & Cook, D. J. (2002) · Cited by 1,196
Reporting of Noninferiority and Equivalence Randomized Trials: An Extension of the CONSORT Statement
Piaggio, G., Elbourne, D. R., Altman, D. G., Pocock, S. J., Evans, S. J. W., & CONSORT Group, F. T. (2006) · JAMA · doi:10.1001/jama.295.10.1152 · Cited by 1,153
The STARD Statement for Reporting Studies of Diagnostic Accuracy: Explanation and Elaboration
Bossuyt, P. M., Reitsma, J. B., Bruns, D. E., Gatsonis, C. A., Glasziou, P. P., Irwig, L. M., Moher, D., Rennie, D., de Vet, H. C. W., & Lijmer, J. G. (2003) · Annals of Internal Medicine · doi:10.7326/0003-4819-138-1-200301070-00012-w1 · Cited by 1,124
Systems to rate the strength of scientific evidence.
West, S. L., King, V., Carey, T. S., Lohr, K. N., Mckoy, N. A., Sutton, S. F., & Lux, L. J. (2002) · PubMed · Cited by 923
Barriers to Participation in Randomised Controlled Trials
Ross, S., Grant, A., Counsell, C., Gillespie, W., Russell, I., & Prescott, R. (1999) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(99)00141-9 · Cited by 907
How to read a paper: The basics of evidence-based medicine
(1998) · Air Medical Journal · doi:10.1016/s1067-991x(98)90091-8 · Cited by 726
Reader's guide to critical appraisal of cohort studies: 2. Assessing potential for confounding
Mamdani, M., Sykora, K., Li, P., Normand, S.-L. T., Streiner, D. L., Austin, P. C., Rochon, P. A., & Anderson, G. M. (2005) · BMJ · doi:10.1136/bmj.330.7497.960 · Cited by 646 · cited 1×
“In this situation, the treatment that someone receives is determined by the patient's or clinician's preferences, referral patterns, current treatment paradigms or local policy”
BackgroundSchottenfeld, D. (2002) · American Journal of Epidemiology · doi:10.1093/aje/kwf028 · Cited by 622
Last, J. M. (1989) · Biometrics · doi:10.2307/2531713 · Cited by 569
Subverting Randomization in Controlled Trials
Schulz, K. F. (1995) · JAMA: The Journal of the American Medical Association · doi:10.1001/jama.1995.03530180050029 · Cited by 516 · cited 1×
“The potential for selective enrollment of patients into the study can be one an important source of bias if the group to which individuals will be allocated is known or can be guessed”
BackgroundPublication Bias and Dissemination of Clinical Research
Begg, C. B., & Berlin, J. A. (1989) · JNCI Journal of the National Cancer Institute · doi:10.1093/jnci/81.2.107 · Cited by 440
QUOROM Group, Moher, D., Cook, D. J., Eastwood, S., Olkin, I., Rennie, D., & Stroup, D. F. (2000) · Journal of British Surgery · doi:10.1046/j.1365-2168.2000.01610.x · Cited by 416
SINGER, P. A. (1996) · Europe PMC (PubMed Central) · Cited by 405
Users’ Guides to the Medical Literature: A Manual for Evidence-Based Clinical Practice
Kleinbart, J., & Williams, M. V. (2002) · Evidence Based Mental Health · doi:10.1136/ebmh.5.4.103 · Cited by 345
Assessment of the accuracy of diagnostic tests: the cross-sectional study
Knottnerus, J. A., & Muris, J. W. (2003) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(03)00206-3 · Cited by 312
How to review the evidence: systematic identification and review of the scientific literature
Glasziou, P., Irwig, L., Bain, C., & Colditz, G. A. (2000) · Cited by 293
Should we be performing more randomized controlled trials evaluating surgical operations?
Solomon, M. J., & McLeod, R. S. (1995) · Surgery · doi:10.1016/s0039-6060(05)80359-9 · Cited by 278
Self-citationGood Enough: A Primer on the Analysis and Interpretation of Noninferiority Trials
Kaul, S., & Diamond, G. A. (2006) · Annals of Internal Medicine · doi:10.7326/0003-4819-145-1-200607040-00011 · Cited by 272 · cited 1×
“Equivalence studies require that the degree of outcome difference at which the two treatments will not be considered equivalent be determined in advance”
BackgroundInclusion of patients in clinical trial analysis: the intention‐to‐treat principle
Heritier, S. R., Gebski, V. J., & Keech, A. C. (2003) · Medical Journal of Australia · doi:10.5694/j.1326-5377.2003.tb05627.x · Cited by 270
Teaching critical appraisal skills in healthcare settings
Horsley, T., Hyde, C., Santesso, N., Parkes, J., Milne, R., & Stewart, R. (2011) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd001270.pub2 · Cited by 247
Critical appraisal of epidemiological studies and clinical trials
Elwood, M. (1998) · Cited by 220
Conflict-of-Interest Policies for Investigators in Clinical Trials
Lo, B., Wolf, L. E., & Berkeley, A. (2000) · New England Journal of Medicine · doi:10.1056/nejm200011303432206 · Cited by 218 · cited 1×
“Researchers are in a position to decide which studies will be conducted in their unit, which patients will be invited to participate in a study and whether certain clinical occurrences should be reported as adverse events”
BackgroundCritical Appraisal of Epidemiological Studies and Clinical Trials
Elwood, M. (2007) · doi:10.1093/acprof:oso/9780198529552.001.0001 · Cited by 192
Reader's guide to critical appraisal of cohort studies: 1. Role and design
Rochon, P. A., Gurwitz, J. H., Sykora, K., Mamdani, M., Streiner, D. L., Garfinkel, S., Normand, S.-L. T., & Geoffrey, M. (2005) · BMJ · doi:10.1136/bmj.330.7496.895 · Cited by 185
Normand, S.-L. T., Sykora, K., Li, P., Mamdani, M., Rochon, P. A., & Anderson, G. M. (2005) · BMJ · doi:10.1136/bmj.330.7498.1021 · Cited by 169
How to read a paper. The basics of evidence based medicine
MOSELEY, I. (2001) · Journal of Neurology, Neurosurgery & Psychiatry · doi:10.1136/jnnp.71.5.709a · Cited by 131
On the shoulders of giants: the great works of physics and astronomy
(2003) · Choice Reviews Online · doi:10.5860/choice.40-6383 · Cited by 123
Growth and decentralization of the medical literature: implications for evidence-based medicine.
Druss, B. G., & Marcus, S. C. (2005) · Europe PMC (PubMed Central) · Cited by 117
Kim, S. Y. H., Millard, R. W., Nisbet, P., Cox, C., & Caine, E. D. (2004) · Journal of Medical Ethics · doi:10.1136/jme.2002.001461 · Cited by 115
The Pocket Guide to Critical Appraisal: A Handbook for Health Care Professionals
Crombie, I. K. (1996) · Medical Entomology and Zoology · Cited by 89
Komesaroff, P. A., & Kerridge, I. (2002) · Medical Journal of Australia · doi:10.5694/j.1326-5377.2002.tb04318.x · Cited by 84
Assessing the Quality of Reports of Systematic Reviews: The QUOROM Statement Compared to Other Tools
Shea, B., Dubé, C., & Moher, D. (2001) · Systematic Reviews in Health Care · doi:10.1002/9780470693926.ch7 · Cited by 80
Critical appraisal for public health: A new checklist
Heller, R. F., Verma, A., Gemmell, I., Harrison, R. S., Hart, J., & Edwards, R. (2007) · Public Health · doi:10.1016/j.puhe.2007.04.012 · Cited by 76
Economics Notes: Types of economic evaluation
Palmer, S., Byford, S., & Raftery, J. (1999) · BMJ · doi:10.1136/bmj.318.7194.1349 · Cited by 76
Sharma, S. (2002) · Evidence-Based Eye Care · doi:10.1097/00132578-200210000-00002 · Cited by 75
The Pocket Guide to Critical Appraisal: A Handbook for Health Care Professionals
Harvey, B. J. (1997) · Europe PMC (PubMed Central) · Cited by 74
Bioethics for clinicians: 17. Conflict of interest in research, education and patient care.
Lemmens, T., & SINGER, P. A. (1998) · Europe PMC (PubMed Central) · Cited by 65
Assessing bias in case-control studies. Proper selection of cases and controls.
Sutton-Tyrrell, K. (1991) · Stroke · doi:10.1161/01.str.22.7.938 · Cited by 61
Palmer, S., Byford, S., & Raftery, J. (1999) · BMJ · doi:10.1136/bmj.318.7194.1349-a · Cited by 57
Randomised controlled trial of the READER method of critical appraisal in general practice
MacAuley, D., McCrum, E. E., & Brown, C. (1998) · BMJ · doi:10.1136/bmj.316.7138.1134 · Cited by 31 · cited 1×
“The criteria used to assess the validity and relevance of scientific literature are not static; they must evolve with improvements in understanding of the important sources of bias inherent in different study designs, and increased awareness of the potential influence of other nonmethodological factors, such as confli…”
MethodInformation overload: what's behind it, what's beyond it?
Glasziou, P. (2008) · The Medical Journal of Australia · doi:10.5694/j.1326-5377.2008.tb01922.x · Cited by 30
READER: an acronym to aid critical reading by general practitioners.
MacAuley, D. (1994) · Europe PMC (PubMed Central) · Cited by 26
Economics notes: Economic evaluation: an introduction
Raftery, J. P. (1998) · BMJ · doi:10.1136/bmj.316.7136.1013 · Cited by 25
Are claims of equivalency in digestive diseases trials supported by the evidence?☆
Tinmouth, J. M., Steele, L. S., Tomlinson, G., & Glazier, R. H. (2004) · Gastroenterology · doi:10.1053/j.gastro.2004.03.005 · Cited by 25
Improving the evidence‐base in surgery: evaluating surgical effectiveness
Young, J. M., & Solomon, M. J. (2003) · ANZ Journal of Surgery · doi:10.1046/j.1445-1433.2003.02662.x · Cited by 23
Self-citationImproving the evidence base in surgery: sources of bias in surgical studies
Young, J. M., & Solomon, M. J. (2003) · ANZ Journal of Surgery · doi:10.1046/j.1445-1433.2003.02611.x · Cited by 23
Self-citationLessons Learned from a Prospective Meta‐analysis
Margitić, S. E., Morgan, T. M., Sager, M. A., & Furberg, C. D. (1995) · Journal of the American Geriatrics Society · doi:10.1111/j.1532-5415.1995.tb05820.x · Cited by 19
Research, ethics and conflicts of interest.
Little, M. (1999) · Journal of Medical Ethics · doi:10.1136/jme.25.3.259 · Cited by 19
Sources of bias in diagnostic accuracy studies and the diagnostic process
Furukawa, T. A., & Guyatt, G. H. (2006) · Canadian Medical Association Journal · doi:10.1503/cmaj.060014 · Cited by 14
Rothman, K. J., Huybrechts, K. F., & Murray, E. J. (2024) · doi:10.1093/oso/9780197751541.001.0001 · Cited by 12
Cited by
Internet Health Information Seeking and the Patient-Physician Relationship: A Systematic Review
Tan, S. S.-L., & Goonawardene, N. (2017) · Journal of Medical Internet Research · doi:10.2196/jmir.5729 · Cited by 1,101
“We assessed quantitative studies using a customized coding scheme that consists of 14 questions adapted from 3 well-established quality assessment tools used for quantitative studies (US National Heart, Lung, and Blood Institute quality assessment tool for observational, cohort, and cross-sectional studies [20], and q…”
The Carbon Footprint of Surgical Operations
Rizan, C., Steinbach, I., Nicholson, R., Lillywhite, R., Reed, M., & Bhutta, M. F. (2020) · Annals of Surgery · doi:10.1097/sla.0000000000003951 · Cited by 400
“Finally, we evaluated the quality of each study making reference to relevant guidelines and critical appraisal tools.(29,30) Studies were appraised independently by 2 researchers (CR, IS) using the system detailed in Supplementary Table 2, http://links.”
MethodMartinez-Gutierrez, M., & Ruiz-Saenz, J. (2016) · BMC Veterinary Research · doi:10.1186/s12917-016-0702-z · Cited by 225
“Finally, database selection could be introducing some bias in the selection of the studies [80]; however, such bias is accepted due to the importance and worldwide relevance of the selected databases.”
BackgroundDunne, S. S., & Dunne, C. P. (2015) · BMC Medicine · doi:10.1186/s12916-015-0415-3 · Cited by 190
“While no papers were excluded based on a subjective assessment of the quality of the reports – as per criteria described in the journal Nature Clinical Practice [29] (specifically, the recommended criteria include: relevance of the paper to the topic; whether the paper contributes new knowledge; type of research quest…”
BackgroundCritical Appraisal of Mixed Methods Studies
Heyvaert, M., Hannes, K., Maes, B., & Onghena, P. (2013) · Journal of Mixed Methods Research · doi:10.1177/1558689813479449 · Cited by 168
“…2010), nor can the use of a generic critical appraisal instrument suffice (cf. Katrak, Bialocerkowski, Massy-Westropp, Kumar, & Grimmer, 2004; Young & Solomon, 2009), that does not stress the importance of appropriately mixing the qualitative and quantitative research elements and of…”
Background
Metadata
- Authors
- Young, J. M., & Solomon, M. J.
- Affiliations
- Sydney South West Area Health Service (AU)
- Corresponding author
- Young, Jane M
- Venue
- Nature Clinical Practice Gastroenterology & Hepatology
- Published
- 2009-01-20
- Volume
- 6
- Issue
- 2
- Pages
- 82-91
- ISSN
- 1743-4378 (ISSN-L)
- 1743-4386
- OpenAlex
- W2166572107
- Cited by
- 178
- References
- 67
- Citations by year
- 2026: 5 · 2025: 6 · 2024: 15 · 2023: 14 · 2022: 16 · 2021: 15 · 2020: 11 · 2019: 14 · 2018: 12 · 2017: 18 · 2016: 10 · 2015: 13 · 2014: 9 · 2013: 12 · 2012: 3
- FWCI
- 0.18
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- www.springer.com/tdm (tdm)
- Locations
- Nature Clinical Practice Gastroenterology & Hepatology (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Health Sciences › Health Professions › General Health Professions › Health Sciences Research and Education
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Keywords
- critical appraisal, evidence-based medicine, research validity, study design critique, statistical analysis, conflicts of interest, clinical decision-making, research quality assessment
- MeSH terms
- Decision Making, Humans, Evidence-Based Medicine
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.