Rating the certainty in evidence in the absence of a single estimate of effect
Open accessAbstract
When studies measure or report outcomes differently, it may not be feasible to pool data across studies to generate a single effect estimate (ie, perform meta-analysis). Instead, only a narrative summary of the effect across different studies might be available. Regardless of whether a single pooled effect estimate is generated or whether data are summarised narratively, decision makers need to know the certainty in the evidence in order to make informed decisions. In this guide, we illustrate how to apply the constructs of the GRADE (Grading of Recommendation, Assessment, Development and Evaluation) approach to assess the certainty in evidence when a meta-analysis has not been performed and data were summarised narratively.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The GRADE system, adapted for narrative synthesis, was used to assess the strength of evidence contributing to effectiveness outcomes.
“The Grading of Recommendations Assessment, Development and Evaluation (GRADE) system,28 adapted for narrative synthesis,29 was used to assess the strength of the evidence contributing to effectiveness outcomes.”
From Approaches to improving mental healthcare for autistic people: Systematic review (2024) · cites this paper as [29]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
GRADE guidelines: 3. Rating the quality of evidence
Balshem, H., Helfand, M., Schünemann, H. J., Oxman, A. D., Kunz, R., Brozek, J., Vist, G. E., Falck-Ytter, Y., Meerpohl, J., & Norris, S. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2010.07.015 · Cited by 8,285 · see citing works
Self-citationGRADE guidelines 6. Rating the quality of evidence—imprecision
Guyatt, G. H., Oxman, A. D., Kunz, R., Brozek, J., Alonso-Coello, P., Rind, D., Devereaux, P. J., Montori, V. M., Freyschuss, B., Vist, G., et al. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2011.01.012 · Cited by 2,693 · see citing works
Self-citationPopay, J., Roberts, H., Sowden, A., Petticrew, M., Arai, L., Rodgers, M., Britten, N., Roen, K., & Duffy, S. (2006) · Lancaster University · doi:10.13140/2.1.1018.4643 · Cited by 2,070 · see citing works
Patient engagement in research: a systematic review
Domecq, J. P., Prutsky, G., Elraiyah, T., Wang, Z., Nabhan, M., Shippee, N., Brito, J. P., Boehmer, K., Hasan, R., Firwana, B., et al. (2014) · BMC Health Services Research · doi:10.1186/1472-6963-14-89 · Cited by 1,773 · see citing works
Self-citationAlonso-Coello, P., Schünemann, H. J., Moberg, J., Brignardello-Petersen, R., Akl, E. A., Davoli, M., Treweek, S., Mustafa, R. A., Rada, G., Rosenbaum, S., et al. (2016) · BMJ · doi:10.1136/bmj.i2016 · Cited by 1,317 · see citing works
Self-citationBourbeau, J. (2003) · Archives of Internal Medicine · doi:10.1001/archinte.163.5.585 · Cited by 933 · see citing works
Lewin, S., Glenton, C., Munthe-Kaas, H., Carlsen, B., Colvin, C. J., Gülmezoglu, M., Noyes, J., Booth, A., Garside, R., & Rashidian, A. (2015) · PLOS Medicine · doi:10.1371/journal.pmed.1001895 · Cited by 866 · see citing works
RAMESES publication standards: meta-narrative reviews
Wong, G., Greenhalgh, T., Westhorp, G., Buckingham, J., & Pawson, R. (2013) · BMC Medicine · doi:10.1186/1741-7015-11-20 · Cited by 610 · see citing works
Self-management education for patients with chronic obstructive pulmonary disease
Effing, T., Monninkhof, E. E., van der Valk, P. P. D. L. P. M., Zielhuis, G. G., Walters, E. H., van der Palen, J. J., & Zwerink, M. (2007) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd002990.pub2 · Cited by 536 · see citing works
Creating a Patient-Centered Health Care Delivery System
Mohammed, K. S., Nolan, M. B., Rajjo, T. I., Shah, N. D., Prokop, L. J., Varkey, P., & Murad, M. H. (2014) · American Journal of Medical Quality · doi:10.1177/1062860614545124 · Cited by 173 · see citing works
Self-citationEvaluation of a self-management plan for chronic obstructive pulmonary disease
Watson, P., Town, G., Holbrook, N., Dwan, C., Toop, L., & Drennan, C. (1997) · European Respiratory Journal · doi:10.1183/09031936.97.10061267 · Cited by 132 · see citing works
Managing Chronic Obstructive Pulmonary Disease in the Community
Boxall, A.-M., Barclay, L., Sayers, A., & Caplan, G. A. (2005) · Journal of Cardiopulmonary Rehabilitation · doi:10.1097/00008483-200511000-00012 · Cited by 126 · see citing works
Part 3. Humanistic Outcomes in the Hypertension and COPD Arms of a Multicenter Outcomes Study
Gourley, G. A., Portner, T. S., Gourley, D. R., Rigolosi, E. L. M., Holt, J. M., Solomon, D. K., Bass, G. E., Wicke, W. R., & Braden, R. L. (1998) · Journal of the American Pharmaceutical Association (1996) · doi:10.1016/s1086-5802(16)30372-2 · Cited by 80 · see citing works
Monninkhofe, E., van der Valk, P., Schermer, T. R., van der Palen, J. A. M., van Herwaarden, C., & Zielhuis, G. G. (2004) · Chronic Respiratory Disease · doi:10.1191/1479972304cd005oa · Cited by 45 · see citing works
Health assessment of commercial drivers: a meta-narrative systematic review
Dabrh, A. M. A., Firwana, B., Cowl, C. T., Steinkraus, L. W., Prokop, L. J., & Murad, M. H. (2014) · BMJ Open · doi:10.1136/bmjopen-2013-003434 · Cited by 45 · see citing works
Self-citationUsing GRADE to respond to health questions with different levels of urgency.
Thayer, K., & Schunemann, H. (2016) · Environment International · doi:10.1016/j.envint.2016.03.027 · Cited by 32 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Guidelines for reporting meta-epidemiological methodology research
Murad, M. H., & Wang, Z. (2017) · Evidence Based Medicine · doi:10.1136/ebmed-2017-110713 · Cited by 515 · see citing works
Using systematic reviews in guideline development: The GRADE approach
Zhang, Y., Akl, E. A., & Schünemann, H. J. (2019) · Research Synthesis Methods · doi:10.1002/jrsm.1313 · Cited by 260 · see citing works
Self-citationChuruangsuk, C., Hall, J., Reynolds, A., Griffin, S. J., Combet, E., & Lean, M. E. J. (2022) · Diabetologia · doi:10.1007/s00125-021-05577-2 · Cited by 204 · see citing works
Hasan, B., Thompson, W. G., Almasri, J., Wang, Z., Lakis, S., Prokop, L. J., Hensrud, D. D., Frie, K. S., Wirtz, M. J., Murad, A. L., et al. (2019) · BMC Nutrition · doi:10.1186/s40795-019-0293-8 · Cited by 162 · see citing works
Self-citationLong distance airborne transmission of SARS-CoV-2: rapid systematic review
Duval, D., Palmer, J. C., Tudge, I., Pearce-Smith, N., O’Connell, E., Bennett, A., & Clark, R. (2022) · BMJ · doi:10.1136/bmj-2021-068743 · Cited by 120 · see citing works
Mudiyanselage, K. W. W., De Santis, K. K., Jörg, F., Saleem, M., Stewart, R., Zeeb, H., & Busse, H. (2024) · BMC Public Health · doi:10.1186/s12889-023-17417-6 · Cited by 48 · see citing works
Approaches to improving mental healthcare for autistic people: Systematic review
Loizou, S., Pemovska, T., Stefanidou, T., Foye, U., Cooper, R., Kular, A., Greenburgh, A., Baldwin, H., Griffiths, J., Saunders, K. R. K., et al. (2024) · BJPsych Open · doi:10.1192/bjo.2024.707 · Cited by 22 · see citing works
“synthesis (28) was used to assess the strength of the evidence contributing to effectiveness”
TAY, Y. N., NEO, N. W. S., & SIAH, C. J. R. (2026) · Nurse Education Today · doi:10.1016/j.nedt.2026.107322 · Cited by 4 · see citing works
- Authors
- Murad, M. H., Mustafa, R. A., Schünemann, H. J., Sultan, S., & Santesso, N.
- Affiliations
- Mayo Clinic (US)
- University of Missouri–Kansas City (US)
- McMaster University (CA)
- University of Minnesota (US)
- Corresponding author
- Murad, M Hassan
- Venue
- Evidence Based Medicine
- Publisher
- BMJ
- Published
- 2017-03-20
- Volume
- 22
- Issue
- 3
- Pages
- 85-87
- ISSN
- 1356-5524 (ISSN-L)
- 1473-6810
- PMC
- PMC5502230
- OpenAlex
- W2603520596
- Cited by
- 717
- References
- 16
- Citations by year
- 2026: 137 · 2025: 110 · 2024: 91 · 2023: 91 · 2022: 125 · 2021: 95 · 2020: 36 · 2019: 20 · 2018: 7 · 2017: 5
- FWCI
- 15.86
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by-nc
- Locations
- BMJ Evidence-Based Medicine (publishedVersion, cc-by-nc)
- PubMed Central (publishedVersion, cc-by-nc)
- PubMed (publishedVersion)
- HighWire Press Open Archive (submittedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Clinical practice guidelines implementation
- Social Sciences › Sociology and Political Science › Delphi Technique in Research
- Keywords
- GRADE, evidence synthesis, clinical decision-making
- MeSH terms
- Clinical Decision-Making, Decision Making, Health Policy, Humans, Guidelines as Topic, Evidence-Based Medicine, Uncertainty
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
