Protocol for a systematic review of the use of qualitative comparative analysis for evaluative questions in public health research
Open accessAbstract
Background: There is an increasing recognition that health intervention research requires methods and approaches that can engage with the complexity of systems, interventions, and the relations between systems and interventions. One approach which shows promise to this end is qualitative comparative analysis (QCA), which examines casual complexity across a medium to large number of cases (between 10 and 60+), whilst also being able to generalise across those cases. Increasingly, QCA is being adopted in public health intervention research. However, there is a limited understanding of how it is being adopted. This systematic review will address this gap, examining how it is being used to understand complex causation; for what settings, populations and interventions; and with which datasets to describe cases. Methods: We will include published and peer-reviewed studies of any public health intervention where the effects on population health, health equity, or intervention uptake are being evaluated. Electronic searches of PubMed, Scopus, Web of Science (incorporating Social Sciences Citation Index and Arts & Humanities Citation Index), Microsoft Academic, and Google Scholar will be performed. This will be supplemented with reference citation tracking and personal contact with experts to identify any additional published studies. Search results will be single screened, with machine learning used to check these results, acting as a ‘second screener’. Any disagreement will be resolved through discussion. Data will be extracted from full texts of eligible studies, which will be assessed against inclusion criteria, and synthesised narratively, using thematic synthesis methods. Discussion: This systematic review will provide an important map of the increasing use of QCA in public health intervention literature. This review will identify the current scope of research in this area, as well as assessing claims about the utility of the method for addressing complex causation in public health research. We will identify implications for better reporting of QCA methods in public health research and for reporting of case studies such that they can be used in future QCA studies. Systematic review registration: PROSPERO, CRD42019131910
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
A protocol was prepared in accordance with the PRISMA-P 2015 statement and published in 2019.
“A protocol was prepared in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analysis Protocols (PRISMA-P) 2015 statement [28], and published in 2019 [27], where the methods are explained in detail.”
From The use of Qualitative Comparative Analysis (QCA) to address causality in complex systems: a systematic review of research on public health interventions (2021) · cites this paper as [27]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Shamseer, L., Moher, D., Clarke, M., Ghersi, D., Liberati, A., Petticrew, M., Shekelle, P., Stewart, L. A., & the PRISMA-P Group (2015) · BMJ · doi:10.1136/bmj.g7647 · Cited by 13,120 · see citing works
Self-citationMethods for the thematic synthesis of qualitative research in systematic reviews
Thomas, J., & Harden, A. (2008) · BMC Medical Research Methodology · doi:10.1186/1471-2288-8-45 · Cited by 9,312 · see citing works
Self-citationProcess evaluation of complex interventions: Medical Research Council guidance
Moore, G. F., Audrey, S., Barker, M., Bond, L., Bonell, C., Hardeman, W., Moore, L., O'Cathain, A., Tinati, T., Wight, D., et al. (2015) · BMJ · doi:10.1136/bmj.h1258 · Cited by 6,204 · see citing works
Ragin, C. C. (2008) · University of Chicago Press eBooks · doi:10.7208/chicago/9780226702797.001.0001 · Cited by 4,595 · see citing works
Redesigning Social Inquiry: Fuzzy Sets and Beyond
Ragin, C. C. (2008) · Cited by 3,398 · see citing works
Set-Theoretic Methods for the Social Sciences: A Guide to Qualitative Comparative Analysis
Schneider, C. Q., & Wagemann, C. (2012) · doi:10.1017/cbo9781139004244 · Cited by 2,862 · see citing works
The Comparative Method: Moving Beyond Qualitative and Quantitative Strategies
Ragin, C. C. (2014) · doi:10.1525/9780520957350 · Cited by 2,130 · see citing works
Methods for the synthesis of qualitative research: a critical review
Barnett-Page, E., & Thomas, J. (2009) · BMC Medical Research Methodology · doi:10.1186/1471-2288-9-59 · Cited by 2,087 · see citing works
Self-citationSet-Theoretic Methods for the Social Sciences: A Guide to Qualitative Comparative Analysis
Schneider, C. Q., & Wagemann, C. (2012) · Cited by 1,345 · see citing works
The need for a complex systems model of evidence for public health
Rutter, H., Savona, N., Glonti, K., Bibby, J., Cummins, S., Finegood, D. T., Greaves, F., Harper, L., Hawe, P., Moore, L., et al. (2017) · The Lancet · doi:10.1016/s0140-6736(17)31267-9 · Cited by 1,260 · see citing works
Self-citationCriteria for evaluating evidence on public health interventions
Rychetnik, L., Frommer, M., Hawe, P., & Shiell, A. (2002) · Journal of Epidemiology and Community Health · doi:10.1136/jech.56.2.119 · Cited by 1,042 · see citing works
Studying complexity in health services research: desperately seeking an overdue paradigm shift
Greenhalgh, T., & Papoutsi, C. (2018) · BMC Medicine · doi:10.1186/s12916-018-1089-4 · Cited by 962 · see citing works
O’Mara-Eves, A., Thomas, J., McNaught, J., Miwa, M., & Ananiadou, S. (2015) · Systematic Reviews · doi:10.1186/2046-4053-4-5 · Cited by 681 · see citing works
Self-citationComplex interventions or complex systems? Implications for health economic evaluation
Shiell, A., Hawe, P., & Gold, L. H. (2008) · BMJ · doi:10.1136/bmj.39569.510521.ad · Cited by 630 · see citing works
Craig, P., Di Ruggiero, E., Frohlich, K. L., Mykhalovskiy, E., White, M., on behalf of the Canadian Institutes of Health Research (CIHR)–National Institute for Health Research (NIHR) Context Guidance Authors Group (listed alphabetically), Campbell, R., Cummins, S., Edwards, N., Hunt, K., et al. (2018) · doi:10.3310/cihr-nihr-01 · Cited by 370 · see citing works
Self-citationO’Mara-Eves, A., Thomas, J., McNaught, J., Miwa, M., & Ananiadou, S. (2015) · Systematic Reviews · doi:10.1186/s13643-015-0031-5 · Cited by 351 · see citing works
Self-citationSchneider, C. Q., & Wagemann, C. (2010) · Comparative Sociology · doi:10.1163/156913210x12493538729838 · Cited by 308 · see citing works
Using qualitative comparative analysis to study causal complexity.
Ragin, C. C. (1999) · PubMed · Cited by 248 · see citing works
Hartmann‐Boyce, J., Bianchi, F., Piernas, C., et al. (2018) · American Journal of Clinical Nutrition · doi:10.1093/ajcn/nqy045 · Cited by 248 · see citing works
Entangled complexity: Why complex interventions are just not complicated enough
Cohn, S. R., Clinch, M., Bunn, C., & Stronge, P. (2013) · Journal of Health Services Research & Policy · doi:10.1258/jhsrp.2012.012036 · Cited by 141 · see citing works
Using qualitative comparative analysis to understand and quantify translation and implementation
Kane, H., Lewis, M. A., Williams, P. A., & Kahwati, L. C. (2014) · Translational Behavioral Medicine · doi:10.1007/s13142-014-0251-6 · Cited by 124 · see citing works
Complexity, Generality, and Qualitative Comparative Analysis
Ragin, C. C., Shulman, D., Weinberg, A. S., et al. (2003) · Field Methods · doi:10.1177/1525822x03257689 · Cited by 121 · see citing works
Cooper, B., & Glaesser, J. (2015) · International Journal of Social Research Methodology · doi:10.1080/13645579.2015.1033799 · Cited by 95 · see citing works
Kelly, M. P., & Federica Russo, B. (2017) · Sociology of Health & Illness · doi:10.1111/1467-9566.12621 · Cited by 78 · see citing works
Warren, J., Wistow, J., & Bambra, C. L. (2013) · Policy and Society · doi:10.1016/j.polsoc.2013.10.002 · Cited by 38 · see citing works
Roberts, M. C., Murphy, T., Moss, J. L., et al. (2018) · American Journal of Public Health · doi:10.2105/ajph.2017.304263 · Cited by 33 · see citing works
Hanckel, B., Ruta, D. A., Scott, G., Peacock, J. L., & Green, J. (2019) · BMC Public Health · doi:10.1186/s12889-019-7511-9 · Cited by 31 · see citing works
Self-citationEvaluating a community-based public health intervention using a complex systems approach
Matheson, A., Walton, M., Gray, R. J., Lindberg, K., Shanthakumar, M., Fyfe, C., Wehipeihana, N., & Borman, B. (2017) · Journal of Public Health · doi:10.1093/pubmed/fdx117 · Cited by 29 · see citing works
van der Kleij, R., Crone, M. R., Paulussen, T. G., van de Gaar, V. M., & Reis, R. (2015) · BMC Public Health · doi:10.1186/s12889-015-2306-0 · Cited by 18 · see citing works
Kane, H., Hinnant, L. W., Day, K. L., Tzeng, J., Soler, R. E., Chambard, M. L., Roussel, A. E., & Heirendt, W. (2016) · Journal of Public Health Management and Practice · doi:10.1097/phh.0000000000000449 · Cited by 18 · see citing works
QCA is of questionable value for policy research
Tanner, S. (2014) · Policy and Society · doi:10.1016/j.polsoc.2014.08.003 · Cited by 16 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Hanckel, B., Petticrew, M., Thomas, J., & Green, J. (2021) · BMC Public Health · doi:10.1186/s12889-021-10926-2 · Cited by 147 · see citing works
Self-citation
- Authors
- Hanckel, B., Petticrew, M., Thomas, J., & Green, J.
- Affiliations
- King's College London (GB)
- London School of Hygiene & Tropical Medicine (GB)
- University College London (GB)
- Corresponding author
- Hanckel, Benjamin
- Venue
- Systematic Reviews
- Published
- 2019-11-01
- Volume
- 8
- Issue
- 1
- Pages
- 252-252
- ISSN
- 2046-4053 (ISSN-L)
- PMC
- PMC6824055
- OpenAlex
- W2984722461
- Cited by
- 10
- References
- 31
- Citations by year
- 2024: 2 · 2023: 2 · 2022: 2 · 2021: 2 · 2020: 1 · 2019: 1
- FWCI
- 0.38
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- Systematic Reviews (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- UCL Discovery (University College London) (acceptedVersion)
- Research Portal (King's College London) (acceptedVersion, other-oa)
- LSHTM Research Online (London School of Hygiene and Tropical Medicine) (acceptedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- Medical Research Council (MR/R014094/1; MR/S014632/1)
- Topics
- Social Sciences › Sociology and Political Science › Qualitative Comparative Analysis Research
- Social Sciences › Decision Sciences › Management Science and Operations Research › Evaluation and Performance Assessment
- Health Sciences › Health Professions › General Health Professions › Health Policy Implementation Science
- Keywords
- qualitative comparative analysis, public health interventions, intervention evaluation, health equity, intervention uptake, population health, methodological reporting, causal complexity
- MeSH terms
- Health Equity, Population Health, Systematic Reviews as Topic, Health Promotion, Health Services Research, Humans, Public Health
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
