Appraising Qualitative Research for Evidence Syntheses: A Compendium of Quality Appraisal Tools
Abstract
As the movement toward evidence-based health policy continues to emphasize the importance of including patient and public perspectives, syntheses of qualitative health research are becoming more common. In response to the focus on independent assessments of rigor in these knowledge products, over 100 appraisal tools for assessing the quality of qualitative research have been developed. The variety of appraisal tools exhibit diverse methods and purposes, reflecting the lack of consensus as to what constitutes appropriate quality criteria for qualitative research. It is a daunting task for those without deep familiarity of the field to choose the best appraisal tool for their purpose. This article provides a description of the structure, content, and objectives of existing appraisal tools for those wanting to evaluate primary qualitative research for a qualitative evidence synthesis. We then discuss common features of appraisal tools and examine their implications for evidence synthesis.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
We selected the Critical Appraisal Skills Programme (CASP) Qualitative Checklist as the most common appraisal tool in QRS that is domain-agnostic, short, and relatively easy to employ.
“Following Majid and Vanstone's [2018] decision tool, we selected the Critical Appraisal Skills Programme (CASP) Qualitative Checklist as (1) the most common appraisal tool in QRS that is (2) domain-agnostic, (3) short, and (4) relatively easy to employ.”
From Generative AI in Game Development: A Qualitative Research Synthesis (2026) · cites this paper as Majid and Vanstone's [2018]
Cites
Standards for Reporting Qualitative Research: A Synthesis of Recommendations
O’Brien, B. C., Harris, I. B., Beckman, T. J., Reed, D. A., & Cook, D. A. (2014) · Academic Medicine · doi:10.1097/acm.0000000000000388 · Cited by 12,211 · see citing works
Methods for the thematic synthesis of qualitative research in systematic reviews
Thomas, J., & Harden, A. (2008) · BMC Medical Research Methodology · doi:10.1186/1471-2288-8-45 · Cited by 9,312 · see citing works
Checklists for improving rigour in qualitative research: a case of the tail wagging the dog?
Barbour, R. S. (2001) · BMJ · doi:10.1136/bmj.322.7294.1115 · Cited by 2,613 · see citing works
Lockwood, C., Munn, Z., & Porritt, K. (2015) · International Journal of Evidence-Based Healthcare · doi:10.1097/xeb.0000000000000062 · Cited by 2,456 · see citing works
Negotiating the swamp: the opportunity and challenge of reflexivity in research practice
Finlay, L. (2002) · Qualitative Research · doi:10.1177/146879410200200205 · Cited by 1,979 · cited 3× · see citing works
“The inappropriate application of reflexive thinking has notable implications for its use as a quality criterion because it is “akin to entering uncertain terrain where solid ground can all too easily give away to swamp and mire” (Finlay, 2002, p. 212).”
BackgroundInfluentialGiacomini, M. K., Cook, D. J., & for the Evidence-Based Medicine Working Group (2000) · JAMA · doi:10.1001/jama.284.3.357 · Cited by 931 · see citing works
Lewin, S., Glenton, C., Munthe-Kaas, H., Carlsen, B., Colvin, C. J., Gülmezoglu, M., Noyes, J., Booth, A., Garside, R., & Rashidian, A. (2015) · PLOS Medicine · doi:10.1371/journal.pmed.1001895 · Cited by 866 · see citing works
The problem of appraising qualitative research
Dixon-Woods, M. (2004) · Quality and Safety in Health Care · doi:10.1136/qshc.2003.008714 · Cited by 653 · see citing works
Finding the Findings in Qualitative Studies
Sandelowski, M., & Barroso, J. (2002) · Journal of Nursing Scholarship · doi:10.1111/j.1547-5069.2002.00213.x · Cited by 468 · see citing works
Toward an agenda for evaluation of qualitative research
Stige, B., Malterud, K., & Midtgarden, T. (2009) · Qualitative Health Research · doi:10.1177/1049732309348501 · Cited by 327 · see citing works
Aggregating Qualitative Findings: An Approach to Theory Development
Estabrooks, C. A., Field, P. A., & Morse, J. M. (1994) · Qualitative Health Research · doi:10.1177/104973239400400410 · Cited by 261 · see citing works
Carroll, C., Booth, A., & Lloyd-Jones, M. (2012) · Qualitative Health Research · doi:10.1177/1049732312452937 · Cited by 248 · see citing works
Carroll, C., & Booth, A. (2015) · Research Synthesis Methods · doi:10.1002/jrsm.1128 · Cited by 235 · see citing works
A proposed framework for developing quality assessment tools
Whiting, P., Wolff, R., Mallett, S., Simera, I., & Savović, J. (2017) · Systematic Reviews · doi:10.1186/s13643-017-0604-6 · Cited by 121 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Improving the content validity of the mixed methods appraisal tool: a modified e-Delphi study
Hong, Q. N., Pluye, P., Fàbregues, S., Bartlett, G., Boardman, F., Cargo, M., Dagenais, P., Gagnon, M.-P., Griffiths, F., Nicolau, B., et al. (2019) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2019.03.008 · Cited by 1,082 · see citing works
“For example, two recent literature reviews on tools for qualitative studies analyzed more than 100 tools [19,47].”
BackgroundQualitative research in nursing and health professions regulation
Squires, A., & Dorsen, C. (2018) · Journal of Nursing Regulation · doi:10.1016/s2155-8256(18)30150-9 · Cited by 108 · see citing works
Questions and answers about conducting systematic reviews in sport and exercise psychology
Gunnell, K., Poitras, V. J., & Tod, D. (2020) · International Review of Sport and Exercise Psychology · doi:10.1080/1750984x.2019.1695141 · Cited by 45 · see citing works
Tenaw, L. A., Ngai, F. W., & Lam, K. (2024) · Midwifery · doi:10.1016/j.midw.2024.104128 · Cited by 11 · see citing works
Generative AI in Game Development: A Qualitative Research Synthesis
Ternar, D.-A., Denisova, A., Cunha, J. M., Kultima, A., & Guckelsberger, C. (2026) · Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3791206 · Cited by 6 · see citing works
Generative AI in Game Development: A Qualitative Research Synthesis
Ternar, A., Denisova, A., Cunha, J. M., Kultima, A., & Guckelsberger, C. (2025) · arXiv
- Authors
- Majid, U., & Vanstone, M.
- Affiliations
- McMaster University (CA)
- Corresponding author
- Vanstone, Meredith
- Venue
- Qualitative Health Research
- Publisher
- SAGE Publications
- Published
- 2018-07-26
- Volume
- 28
- Issue
- 13
- Pages
- 2115-2131
- ISSN
- 1049-7323 (ISSN-L)
- 1552-7557
- OpenAlex
- W2884902003
- Cited by
- 301
- References
- 14
- Citations by year
- 2026: 26 · 2025: 44 · 2024: 39 · 2023: 28 · 2022: 47 · 2021: 32 · 2020: 65 · 2019: 17 · 2018: 3
- FWCI
- 22.31
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Qualitative Health Research (publishedVersion)
- PubMed (publishedVersion)
- Funders
- Government of Ontario
- Canadian Institutes of Health Research
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Education › Qualitative Research Methods and Applications
- Social Sciences › Sociology and Political Science › Delphi Technique in Research
- Keywords
- quality assessment tools, qualitative research quality, qualitative evidence synthesis, research rigor, appraisal criteria, evidence-based health policy
- MeSH terms
- Data Accuracy, Humans, Surveys and Questionnaires, Research Design, Meta-Analysis as Topic, Reproducibility of Results, Qualitative Research
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
