The DOI matches its Crossref record; 47 of the 48 references it cites were found in a registry or an index.
Using Qualitative Evidence in Decision Making for Health and Social Interventions: An Approach to Assess Confidence in Findings from Qualitative Evidence Syntheses (GRADE-CERQual)
Open accessAbstract
Simon Lewin and colleagues present a methodology for increasing transparency and confidence in qualitative research synthesis.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Downgrades were primarily driven by methodological limitations related to limited reporting of researcher positioning and reflexivity across contributing studies, and depending on the finding, by context-dependence (relevance) or uneven richness of supporting data (adequacy) (Lewin et al., 2015).”
From Relational and Institutional Disenfranchisement in Suicide Bereavement: A Qualitative Meta-Synthesis of Family Members' Experiences (2026) · cites this paper as Lewin et al., 2015
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“The Grading of Recommendations Assessment, Development, and Evaluation-Confidence in the Evidence from Reviews of Qualitative Research (GRADE-CERQual) approach [37] was used to assess the confidence that can be attributed to the evidence informing each individual review finding, with ratings of high, moderate, or low confidence being attributed to each finding.”
From Older Adults’ Experiences and Perceptions of Immersive Virtual Reality: Systematic Review and Thematic Synthesis (2022) · cites this paper as [37]
The cited source says what the claim attributes to it.
In the citing paper
“Confidence in the evidence was assessed using the Grading of Recommendations Assessment, Development, and Evaluation-Confidence in the Evidence from Reviews of Qualitative Research approach.”
From Older Adults’ Experiences and Perceptions of Immersive Virtual Reality: Systematic Review and Thematic Synthesis (2022) · cites this paper as [37]
The cited source says what the claim attributes to it.
In the citing paper
“Confidence in the evidence was assessed using the GRADE-CERQual [37] tool and ranged from high to moderate (Multimedia Appendix 7 [9,14-16,44-52]).”
From Older Adults’ Experiences and Perceptions of Immersive Virtual Reality: Systematic Review and Thematic Synthesis (2022) · cites this paper as [37]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Tong, A., Sainsbury, P., & Craig, J. (2007) · International Journal for Quality in Health Care · doi:10.1093/intqhc/mzm042 · Cited by 42,509
Dermyshi, E., Wang, Y., Yan, C., Hong, W., Qiu, G., Gong, X., & Zhang, T. (2017) · Neonatology · doi:10.1159/000454668 · Cited by 24,586
GRADE: an emerging consensus on rating quality of evidence and strength of recommendations
Guyatt, G. H., Oxman, A. D., Vist, G. E., Kunz, R., Falck-Ytter, Y., Alonso-Coello, P., & Schünemann, H. J. (2008) · BMJ · doi:10.1136/bmj.39489.470347.ad · Cited by 23,305
How Many Interviews Are Enough?: An Experiment with Data Saturation and Variability
Guest, G., Bunce, A., & Johnson, L. (2006) · Field Methods · doi:10.1177/1525822x05279903 · Cited by 20,808
GRADE guidelines: 1. Introduction—GRADE evidence profiles and summary of findings tables
Guyatt, G., Oxman, A. D., Akl, E. A., Kunz, R., Vist, G., Brozek, J., Norris, S., Falck-Ytter, Y., Glasziou, P., & deBeer, H. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2010.04.026 · Cited by 10,681
Gibbs, G. (2007) · doi:10.4135/9781849208574 · Cited by 4,299
Enhancing the quality and credibility of qualitative analysis.
(1999) · PubMed Central · Cited by 4,133
Systematic reviews in the social sciences: a practical guide
(2006) · Choice Reviews Online · doi:10.5860/choice.43-5664 · Cited by 4,081
Systematic Reviews in the Social Sciences: A Practical Guide
Petticrew, M., & Roberts, H. (2006) · doi:10.1002/9780470754887 · Cited by 3,586
Norderyd, O. M., Koch, G., Papias, A., Köhler, A. A., Helkimo, A. N., Brahm, C., Lindmark, U., Lindfors, N., Mattsson, A., Rolander, B., et al. (2015) · PubMed · Cited by 3,584
What is an adequate sample size? Operationalising data saturation for theory-based interview studies
Francis, J. J., Johnston, M., Robertson, C., Glidewell, L., Entwistle, V., Eccles, M. P., & Grimshaw, J. M. (2010) · Psychology & Health · doi:10.1080/08870440903194015 · Cited by 3,429
Naturalistic inquiry and the saturation concept: a research note
Bowen, G. A. (2008) · Qualitative Research · doi:10.1177/1468794107085301 · Cited by 2,749
Checklists for improving rigour in qualitative research: a case of the tail wagging the dog?
Barbour, R. S. (2001) · BMJ · doi:10.1136/bmj.322.7294.1115 · Cited by 2,613
Methods for the synthesis of qualitative research: a critical review
Barnett-Page, E., & Thomas, J. (2009) · BMC Medical Research Methodology · doi:10.1186/1471-2288-9-59 · Cited by 2,386
Purposeful Sampling in Qualitative Research Synthesis
Suri, H. (2011) · Qualitative Research Journal · doi:10.3316/qrj1102063 · Cited by 2,284
Popay, J., Roberts, H., Sowden, A., Petticrew, M., Arai, L., Rodgers, M., Britten, N., Roen, K., & Duffy, S. (2006) · Lancaster University · doi:10.13140/2.1.1018.4643 · Cited by 2,070
O’Reilly, M., & Parker, N. (2013) · Qualitative Research · doi:10.1177/1468794112446106 · Cited by 1,943
Bohren, M. A., Vogel, J. P., Hunter, E. C., Lutsiv, O., Makh, S. K., Souza, J. P., Aguiar, C., Saraiva Coneglian, F., Diniz, A. L. A., Tunçalp, Ö., et al. (2015) · PLOS Medicine · doi:10.1371/journal.pmed.1001847 · Cited by 1,656 · cited 1×
“The current version of CERQual has been used in one published review [36] and is currently being used in a further ten reviews, at least half of which are being produced to support WHO guidance.”
MethodSystematic Approaches to a Successful Literature Review
Booth, A., Sutton, A., Clowes, M., & James, M. M.-S. (2022) · SAGE Publications Ltd eBooks · doi:10.4135/9781036232009 · Cited by 1,174
Self-citationQualitative research methods in health technology assessment: a review of the literature.
Murphy, Dingwall, Greatbatch, Parker, & Watson (1998) · Health Technology Assessment · doi:10.3310/hta2160 · Cited by 1,107
Clarifying differences between review designs and methods
Gough, D., Thomas, J., & Oliver, S. (2012) · Systematic Reviews · doi:10.1186/2046-4053-1-28 · Cited by 1,063
Real qualitative researchers do not count: The use of numbers in qualitative research
Sandelowski, M. (2001) · Research in Nursing & Health · doi:10.1002/nur.1025 · Cited by 981
Bohren, M. A., Hunter, E. C., Munthe-Kaas, H. M., Souza, J. P., Vogel, J. P., & Gülmezoglu, A. M. (2014) · Reproductive Health · doi:10.1186/1742-4755-11-71 · Cited by 823
Self-citationEstablishing confidence in the output of qualitative research synthesis: the ConQual approach.
Munn, Z., Porritt, K., Lockwood, C., Aromataris, E., & Pearson, A. (2014) · BMC Medical Research Methodology · doi:10.1186/1471-2288-14-108 · Cited by 803
Ponterotto, J. G. (2015) · The Qualitative Report · doi:10.46743/2160-3715/2006.1666 · Cited by 797
Guyatt, G., Oxman, A. D., Sultan, S., Brozek, J., Glasziou, P., Alonso-Coello, P., Atkins, D., Kunz, R., Montori, V., Jaeschke, R., et al. (2013) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2012.01.006 · Cited by 762
Guidance on the conduct of narrative synthesis in sytematic reviews
Popay, J., Roberts, H. M., Sowden, A. J., Petticrew, M. P., Arai, L., Rodgers, M., & Britten, N. (2006) · Cited by 690
The problem of appraising qualitative research
Dixon-Woods, M. (2004) · Quality and Safety in Health Care · doi:10.1136/qshc.2003.008714 · Cited by 653
Dixon-Woods, M., Sutton, A., Shaw, R., Miller, T., Smith, J., Young, B., Bonas, S., Booth, A., & Jones, D. (2007) · Journal of Health Services Research & Policy · doi:10.1258/135581907779497486 · Cited by 546
Self-citationAppraising the quality of qualitative research
Walsh, D., & Downe, S. (2006) · Midwifery · doi:10.1016/j.midw.2005.05.004 · Cited by 523
Glenton, C., Colvin, C., Carlsen, B., Swartz, A., Lewin, S., Noyes, J., & Rashidian, A. (2013) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd010414.pub2 · Cited by 454
Self-citationSystematic Approaches to a Successful Literature Review
Louise, D. (2013) · Nurse Education in Practice · doi:10.1016/j.nepr.2012.12.007 · Cited by 446
Oxman, A. D., Lavis, J. N., Lewin, S. A., & Fretheim, A. (2009) · Health Research Policy and Systems · doi:10.1186/1478-4505-7-s1-s1 · Cited by 399
Self-citationPetticrew, M. P., Rehfuess, E. A., Noyes, J., Higgins, J. P. T., Mayhew, A., Tomás Pantoja, C., Shemilt, I., & Sowden, A. J. (2013) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2013.06.005 · Cited by 328
Self-citationIncluding qualitative research in systematic reviews: opportunities and problems
Dixon‐Woods, M., Fitzpatrick, R. M., & Roberts, K. (2001) · Journal of Evaluation in Clinical Practice · doi:10.1046/j.1365-2753.2001.00257.x · Cited by 298
Evaluating and synthesizing qualitative research: the need to develop a distinctive approach
Barbour, R. S., & Barbour, M. (2003) · Journal of Evaluation in Clinical Practice · doi:10.1046/j.1365-2753.2003.00371.x · Cited by 217
Treweek, S. P., Oxman, A. D., Alderson, P., Bossuyt, P. M., Brandt, L., Brożek, J., Davoli, M., Flottorp, S. A., Harbour, R., Hill, S. R., et al. (2013) · Implementation Science · doi:10.1186/1748-5908-8-6 · Cited by 213
Karimi‐Shahanjarini, A., Shakibazadeh, E., Rashidian, A., Hajimiri, K., Glenton, C., Noyes, J., Lewin, S. A., Laurant, M. G. H., & Colvin, C. J. (2019) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd010412.pub2 · Cited by 180
Self-citation(2003) · UCL Discovery (University College London) · Cited by 162
Desperately Seeking Dissonance
Booth, A., Carroll, C., Ilott, I., Low, L. L., & Cooper, K. L. (2012) · Qualitative Health Research · doi:10.1177/1049732312466295 · Cited by 142
Self-citationSynthesising quantitative and qualitative research in evidence-based patient information
Goldsmith, M. R., Bankhead, C. R., & Austoker, J. (2007) · Journal of Epidemiology and Community Health · doi:10.1136/jech.2006.046110 · Cited by 142
Colvin, C. J., de Heer, J., Winterton, L., Mellenkamp, M., Glenton, C., Noyes, J., Lewin, S., & Rashidian, A. (2013) · Midwifery · doi:10.1016/j.midw.2013.05.001 · Cited by 135
Self-citationRashidian, A., Shakibazadeh, E., Karimi‐Shahanjarini, A., Glenton, C., Noyes, J., Lewin, S. A., Colvin, C. J., & Laurant, M. G. H. (2013) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd010412 · Cited by 127
Self-citationPetticrew, M. P., Egan, M., Thomson, H., Hamilton, V., Kunkler, R., & Roberts, H. M. (2008) · Journal of Epidemiology & Community Health · doi:10.1136/jech.2006.059394 · Cited by 84
Noyes, J., Gough, D., Lewin, S. A., Mayhew, A., Michie, S., Tomás Pantoja, C., Petticrew, M. P., Pottie, K. C., Rehfuess, E. A., Shemilt, I., et al. (2013) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2013.07.003 · Cited by 75
Self-citationLewin, S. A., Bosch‐Capblanch, X., Oliver, S. R., Akl, E. A., Vist, G. E., Lavis, J. N., Ghersi, D., Røttingen, J., Steinmann, P., Gülmezoglu, M., et al. (2012) · PLoS Medicine · doi:10.1371/journal.pmed.1001187 · Cited by 62
Self-citationReviews of Qualitative Evidence: A New Milestone for Cochrane
Gülmezoglu, A. M., Chandler, J., Shepperd, S., & Tomás Pantoja, C. (2013) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.ed000073 · Cited by 32
Flagged first, then by citations
1 of the 48 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
Applying GRADE-CERQual to qualitative evidence synthesis findings: introduction to the series
Lewin, S., Booth, A., Glenton, C., Munthe-Kaas, H., Rashidian, A., Wainwright, M., Bohren, M. A., Tunçalp, Ö., Colvin, C. J., Garside, R., et al. (2018) · Implementation Science · doi:10.1186/s13012-017-0688-3 · Cited by 1,231
Self-citationImproving the content validity of the mixed methods appraisal tool: a modified e-Delphi study
Hong, Q. N., Pluye, P., Fàbregues, S., Bartlett, G., Boardman, F., Cargo, M., Dagenais, P., Gagnon, M.-P., Griffiths, F., Nicolau, B., et al. (2019) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2019.03.008 · Cited by 1,082
“Several approaches for rating the overall quality of evidence have been developed, such as Grading of Recommendations, Assessment, Development and Evaluations (GRADE) [38] and GRADE-Confidence in the Evidence from Reviews of Qualitative research (CERQual) [39].”
MethodRating the certainty in evidence in the absence of a single estimate of effect
Murad, M. H., Mustafa, R. A., Schünemann, H. J., Sultan, S., & Santesso, N. (2017) · Evidence Based Medicine · doi:10.1136/ebmed-2017-110668 · Cited by 717
Ames, H., Glenton, C., & Lewin, S. A. (2019) · BMC Medical Research Methodology · doi:10.1186/s12874-019-0665-4 · Cited by 612
Self-citationLewin, S. A., Bohren, M. A., Rashidian, A., Munthe‐Kaas, H. M., Glenton, C., Colvin, C. J., Garside, R., Noyes, J., Booth, A., Tunçalp, Ö., et al. (2018) · Implementation Science · doi:10.1186/s13012-017-0689-2 · Cited by 606
Self-citation
Metadata
- Authors
- Lewin, S., Glenton, C., Munthe-Kaas, H., Carlsen, B., Colvin, C. J., Gülmezoglu, M., Noyes, J., Booth, A., Garside, R., & Rashidian, A.
- Affiliations
- South African Medical Research Council (ZA)
- Nasjonalt Kunnskapssenter for Helsetjenesten (NO)
- Uni Research (Norway) (NO)
- University of Cape Town (ZA)
- World Bank (US)
- Bangor University (GB)
- University of Sheffield (GB)
- University of Exeter (GB)
- World Health Organization Regional Office for the Eastern Mediterranean (EG)
- Corresponding author
- Lewin, Simon
- ORCID
- Venue
- PLOS Medicine
- Publisher
- Public Library of Science (PLoS)
- Published
- 2015-10-27
- Volume
- 12
- Issue
- 10
- Pages
- e1001895
- ISSN
- 1549-1277 (ISSN-L)
- 1549-1676
- PMC
- PMC4624425
- OpenAlex
- W2107819237
- Cited by
- 866
- References
- 48
- Citations by year
- 2026: 68 · 2025: 60 · 2024: 84 · 2023: 66 · 2022: 77 · 2021: 120 · 2020: 86 · 2019: 95 · 2018: 75 · 2017: 77 · 2016: 54 · 2015: 2
- FWCI
- 63.46
- Type
- journal article
- Language
- en
- Update notices
- correction (2016-06-10)
- Open access
- gold
- doi.org
- Licence
- Locations
- PLoS Medicine (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Bangor University Research Portal (Bangor University) (acceptedVersion)
- White Rose Research Online (University of Leeds, The University of Sheffield, University of York) (acceptedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- Bergen Open Research Archive (BORA) (University of Bergen) (submittedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Figshare (submittedVersion, cc-by)
- Figshare (submittedVersion)
- Open University of Cape Town (University of Cape Town) (submittedVersion, cc-by)
- Open Research Exeter (University of Exeter) (submittedVersion, cc-by)
- Bangor University Research Portal (Bangor University) (submittedVersion, other-oa)
- Funders
- World Health Organization
- South African Medical Research Council
- Alliance for Health Policy and Systems Research
- Medical Research Council
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Education › Qualitative Research Methods and Applications
- Social Sciences › Sociology and Political Science › Delphi Technique in Research
- Keywords
- qualitative evidence synthesis, GRADE-CERQual, decision-making, transparency, research evidence appraisal
- MeSH terms
- Decision Making, Humans, Public Policy, Research Design, Qualitative Research, Evidence-Based Practice
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
