Making a difference: towards a method for weighing the evidence in a qualitative synthesis
The DOI matches its Crossref record; 6 of the 6 references it cites were found in a registry or an index.
Abstract
Objectives In a qualitative synthesis, primary qualitative studies are integrated to develop a theory or evidence‐based interventions. Until now, the strength of the evidence in the primary studies has not been taken into account in the outcome of the qualitative synthesis. In this paper, a method is developed and evaluated to assign weights to the findings of the qualitative studies using both the frequency and the quality of the reported results. Method Seventeen qualitative studies were retrieved in an illustrative example project on children and trauma. Findings were extracted from the primary studies with the use of coding which resulted in 14 themes. The quality of the studies was appraised using both expert judgement and a quality checklist. These outcomes are used to calculate the weights. Results When the outcomes of the checklist appraisal are compared with those of the expert judgement, the effect on the strength of the evidence is virtually always in the same direction. We found that as the frequency with which a concept is studied is low, the strength of the evidence oftentimes decreases even further when using the quality of the results in the weighing process. Conclusions In the end, the outcomes of a critical appraisal affect the weight that is placed on particular studies. The use of a checklist is recommended because of its more distinguishing ability. The method that was developed for assigning weights to the evidence is discussed in light of both the quality appraisal in qualitative research and the objectives of qualitative synthesis.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Systematic Reviews in the Social Sciences: A Practical Guide
Petticrew, M., & Roberts, H. (2006) · doi:10.1002/9780470754887 · Cited by 3,586
Conducting a meta-ethnography of qualitative literature: Lessons learnt
Atkins, S., Lewin, S., Smith, H., Engel, M., Fretheim, A., & Volmink, J. (2008) · BMC Medical Research Methodology · doi:10.1186/1471-2288-8-21 · Cited by 981
Meta-Study of Qualitative Health Research
Paterson, B., Thorne, S., Canam, C., & Jillings, C. (2001) · doi:10.4135/9781412985017 · Cited by 664
The problem of appraising qualitative research
Dixon-Woods, M. (2004) · Quality and Safety in Health Care · doi:10.1136/qshc.2003.008714 · Cited by 653 · cited 1×
“This so-called critique-led approach gives weight to the argument that qualitative research deals mainly with the conceptual contribution and that, as a consequence, all input is welcomed [33].”
BackgroundDixon-Woods, M., Sutton, A., Shaw, R., Miller, T., Smith, J., Young, B., Bonas, S., Booth, A., & Jones, D. (2007) · Journal of Health Services Research & Policy · doi:10.1258/135581907779497486 · Cited by 546
Testimonial Psychotherapy for Adolescent Refugees: a Case Series
Lustig, S. L., Weine, S. M., Saxe, G. N., & Beardslee, W. R. (2004) · Transcultural Psychiatry · doi:10.1177/1363461504041352 · Cited by 82
Cited by
Factors influencing acceptance of technology for aging in place: A systematic review
Peek, S. T. M., Wouters, E. J. M., van Hoof, J., Luijkx, K. G., Boeije, H. R., & Vrijhoef, H. J. M. (2014) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2014.01.004 · Cited by 1,206
Self-citationLong, H. A., French, D. P., & Brooks, J. M. (2020) · Research Methods in Medicine & Health Sciences · doi:10.1177/2632084320947559 · Cited by 1,112
“Best fit” framework synthesis: refining the method
Carroll, C., Booth, A., Leaviss, J., & Rick, J. (2013) · BMC Medical Research Methodology · doi:10.1186/1471-2288-13-37 · Cited by 649
“is affected by the omission of studies with methodological flaws [56] or from certain sources [20].”
BackgroundMethodTheme in Qualitative Content Analysis and Thematic Analysis
Vaismoradi, M., & Snelgrove, S. R. (2019) · Forum qualitative Sozialforschung · doi:10.17169/fqs-20.3.3376 · Cited by 441
Writing Usable Qualitative Health Research Findings
Sandelowski, M., & Leeman, J. (2012) · Qualitative Health Research · doi:10.1177/1049732312450368 · Cited by 418
Metadata
- Authors
- Boeije, H. R., van Wesel, F., & Alisic, E.
- Venue
- Journal of Evaluation in Clinical Practice
- Publisher
- Wiley
- Published
- 2011-04-19
- Volume
- 17
- Issue
- 4
- Pages
- 657-663
- ISSN
- 1356-1294 (ISSN-L)
- 1365-2753
- OpenAlex
- W2105551154
- Cited by
- 67
- References
- 6
- FWCI
- 4.39
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Making a difference: towards a method for weighing the evidence in a qualitative synthesis. https://app.cochat.ai/papers/10.1111/j.1365-2753.2011.01674.x
- The paper
- Boeije, H. R., van Wesel, F., & Alisic, E. (2011). Making a difference: towards a method for weighing the evidence in a qualitative synthesis. Journal of Evaluation in Clinical Practice. https://doi.org/10.1111/j.1365-2753.2011.01674.x
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.