Improving Conduct and Reporting of Narrative Synthesis of Quantitative Data (ICONS-Quant): protocol for a mixed methods study to develop a reporting guideline
Open accessThe DOI matches its Crossref record; 19 of the 19 references it cites were found in a registry or an index.
Abstract
Introduction: Reliable evidence syntheses, based on rigorous systematic reviews, provide essential support for evidence-informed clinical practice and health policy. Systematic reviews should use reproducible and transparent methods to draw conclusions from the available body of evidence. Narrative synthesis of quantitative data (NS) is a method commonly used in systematic reviews where it may not be appropriate, or possible, to meta-analyse estimates of intervention effects. A common criticism of NS is that it is opaque and subject to author interpretation, casting doubt on the trustworthiness of a review’s conclusions. Despite published guidance funded by the UK’s Economic and Social Research Council on the conduct of NS, recent work suggests that this guidance is rarely used and many review authors appear to be unclear about best practice. To improve the way that NS is conducted and reported, we are developing a reporting guideline for NS of quantitative data. Methods: We will assess how NS is implemented and reported in Cochrane systematic reviews and the findings will inform the creation of a Delphi consensus exercise by an expert panel. We will use this Delphi survey to develop a checklist for reporting standards for NS. This will be accompanied by supplementary guidance on the conduct and reporting of NS, as well as an online training resource. Ethics and dissemination: Ethical approval for the Delphi survey was obtained from the University of Glasgow in December 2017 (reference 400170060). Dissemination of the results of this study will be through peer-reviewed publications, and national and international conferences.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The SWiM reporting guideline emerged from a project aiming to improve the transparency and conduct of narrative synthesis (ICONS-Quant: Improving the CONduct and reporting of Narrative Synthesis).10”
From Synthesis without meta-analysis (SWiM) in systematic reviews: reporting guideline (2020) · cites this paper as [10]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The SWiM guideline is intended for use in systematic reviews examining the quantitative effects of interventions for which meta-analysis of effect estimates is not possible, or not appropriate, for a least some outcomes.10”
From Synthesis without meta-analysis (SWiM) in systematic reviews: reporting guideline (2020) · cites this paper as [10]
Experimental — accuracy not yet measuredIn the citing paper
“A protocol for the project is available,10 and the guideline development was registered with the EQUATOR Network, after confirmation that no similar guideline was in development.”
From Synthesis without meta-analysis (SWiM) in systematic reviews: reporting guideline (2020) · cites this paper as [10]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Preferred Reporting Items for Systematic Reviews and Meta-Analyses: the PRISMA statement
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & for the PRISMA Group (2009) · BMJ · doi:10.1136/bmj.b2535 · Cited by 83,823
Preferred reporting items for systematic reviews and meta-analyses: the PRISMA Statement.
(2009) · PubMed Central · Cited by 44,859
Cochrane Handbook for Systematic Reviews of Interventions: Cochrane Book Series
(2008) · doi:10.1002/9780470712184 · Cited by 25,048
Hoffmann, T. C., Glasziou, P. P., Boutron, I., Milne, R., Perera, R., Moher, D., Altman, D. G., Barbour, V., Macdonald, H., Johnston, M., et al. (2014) · BMJ · doi:10.1136/bmj.g1687 · Cited by 10,548
Research guidelines for the Delphi survey technique
Hasson, F. (2000) · Journal of Advanced Nursing · doi:10.1046/j.1365-2648.2000.01567.x · Cited by 5,951
Systematic reviews in the social sciences: a practical guide
(2006) · Choice Reviews Online · doi:10.5860/choice.43-5664 · Cited by 4,089
Diamond, I. R., Grant, R. C., Feldman, B. M., Pencharz, P. B., Ling, S. C., Moore, A. M., & Wales, P. W. (2014) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2013.12.002 · Cited by 3,740
Systematic Reviews in the Social Sciences: A Practical Guide
Petticrew, M., & Roberts, H. (2006) · doi:10.1002/9780470754887 · Cited by 3,586
Popay, J., Roberts, H., Sowden, A., Petticrew, M., Arai, L., Rodgers, M., Britten, N., Roen, K., & Duffy, S. (2006) · Lancaster University · doi:10.13140/2.1.1018.4643 · Cited by 2,070
Self-citationGuidance for Developers of Health Research Reporting Guidelines
Moher, D., Schulz, K. F., Simera, I., & Altman, D. G. (2010) · PLoS Medicine · doi:10.1371/journal.pmed.1000217 · Cited by 1,437 · cited 1×
“Provision of reporting guidelines alone will not necessarily lead to improved research conduct; provision of explanatory guidance, dissemination, endorsement and support for adherence is also necessary.(11) We will produce materials to support the implementation of best practice in the application of NS methods, and i…”
Method
Research works printed but not linked: 3
- Popay J Methods for the development of NICE public health guidance. 3rd edn London: National Institute for Clinical Excellence, 2012. 27905711
- Pope C, Mays N, Popay J Synthesising qualitative and quantitative health evidence: a guide to methods. London: Open University Press, 2007.
- Higgins JP, Green S Cochrane handbook for systematic reviews of interventions: Wiley Online Library, 2011.
Cited by
Items 101–160 of 160
General Surgery 4.0 – A Systematic Review of Extended Reality Interventions in General Surgery
Kowal, M. R., Williams, T., Dosis, A., Pathak, S., Farid, S. G., Stocken, D. D., Lodge, J. P. A., Ali, S., Tolan, D., & Jayne, D. G. (2026) · Surgical Innovation · doi:10.1177/15533506251415440 · Cited by 1
Yang, X., Wu, J., Ma, Y., Li, C., Chen, A., Zheng, K., Sun, S. L., Tang, Y., & Ren, Z. (2026) · Review Journal of Autism and Developmental Disorders · doi:10.1007/s40489-026-00555-z · Cited by 1
The relationship between fermented foods and depression: A systematic review.
Hanssens, M. B., Patterson, S., & Wagner, C. (2019) · SOPHIA (St. Catherine University) · Cited by 1
Do Practices such as Dowry and Bride Prices Relate to Violence Among Migrants in OECD countries?
Yuol, A. (2019) · Social Science Protocols · doi:10.7565/ssp.2019.2569 · Cited by 1
Kvasnak, K., Brunton, G., Lemonde, M., Chyzzy, B., & Dick, J. A. (2021) · International Health Trends and Perspectives · doi:10.32920/ihtp.v1i2.1438 · Cited by 1
Factors Impacting the Recruitment and Retention of Nurses in Nigeria: Systematic Review
Bomide, F. (2024) · African Journal of Health Nursing and Midwifery · doi:10.52589/ajhnm-6e05aykl · Cited by 1
Use of the HCR-20 Version 3 with Women: A Narrative Synthesis
Gunenc, C., Fitton, L., & Beazley, P. I. (2024) · Journal of Forensic Psychology Research and Practice · doi:10.1080/24732850.2024.2386555 · Cited by 1
“A narrative synthesis can be used when statistical synthesis is not possible due to limited available data (e.g. Campbell et al., 2018).”
Qualitative Evidence Synthesis Using COSTAQDA
Costa, K., & Costa, A. P. (2024) · Lecture notes in networks and systems · doi:10.1007/978-3-031-65735-1_4 · Cited by 2
Systematic literature reviews in migration studies: approaches to context-sensitive synthesis
Sommerfelt, T., Mjelva, M. B., Carling, J., & Aslany, M. (2025) · Comparative Migration Studies · doi:10.1186/s40878-025-00502-7 · Cited by 1
“In this tradition, a common critique of narrative synthesis of quantitative research is that it is “opaque and subject to author interpretation, casting doubt on the trustworthiness of a review’s conclusions” (Campbell et al., 2018, p. 1).”
Optimising outcome for Achilles tendinopathy : an exploration into cognitive and contextual factors
Mallows, A. J. (2019) · Open Access at Essex (University of Essex)
Diet choices: psychological impacts and influences
Larcombe, S. J. (2026) · Warwick Research Archive Portal (University of Warwick) · doi:10.82444/warw.33825049
Review for "Systematic review of interventions to improve nurses’ work environments"
(2020) · Publons reviews and discussion · doi:10.1111/jan.14462/v2/review1
Decision letter for "Systematic review of interventions to improve nurses’ work environments"
(2020) · Publons reviews and discussion · doi:10.1111/jan.14462/v2/decision1
(2020) · Publons reviews and discussion · doi:10.1111/jan.14342/v2/review3
(2020) · Publons reviews and discussion · doi:10.1111/jan.14342/v2/decision1
(2019) · Publons reviews and discussion · doi:10.1111/jan.14342/v1/review1
Effects of Ventilator-Associated Pneumonia Among Saudian’s Critically Ill Patients, Systemic Review
Al-Gethami, E. H. M. (2022) · British journal of healthcare and medical research · doi:10.14738/jbemi.93.12434
Al-Gethami, E. H. M., Altalhi, M. A., Alrbai, K. A., Algethami, A. H., & Altalhi, M. A. (2022) · American Journal of Nursing Science · doi:10.11648/j.ajns.20221102.15
Physiotherapy management of nociplastic pain: A Delphi study of Italian specialists
Baroni, A., Margelli, M., Saoncella, A., Toigo, I., Antonellini, G., & Straudi, S. (2023) · Pain Practice · doi:10.1111/papr.13238
“Agreement between closed- ended questions was assessed by analyzing yes and no percentages.(19) Physiotherapist competences that reached high consensus in Round 1 (level of agreement ≥ 50%) were included in the final list of competences.”
MethodUse of Artificial Intelligence Interventions in Chronic Pain Management: A Scoping Review (Preprint)
Abiodun, A. O., Das, S., & Taylor, A. (2024) · Journal of Medical Internet Research · doi:10.2196/57883
Gender-based violence and women’s nutrition: a systematic review
Meyer, S. R., Gillespie, A., Bhatt-Carreño, S., Johnstone, L., & Orjuela‐Grimm, M. (2025) · BMJ Global Health · doi:10.1136/bmjgh-2025-018996
Advances in Supportive Therapy for Patients With Thalassemia: A Systematic Review
Septian, R. B., Rukhama, R. A., & Hasanah, N. N. (2025) · Biological Research For Nursing · doi:10.1177/10998004251406267
“This was carried out by summarizing and explaining the fi ndings from multiple studies in a narrative format, which is often used to discuss diverse types of evidence and complex research questions (Campbell et al., 2018).”
Additional file 1 of Promoting the practice of exclusive breastfeeding: a philosophic scoping review
Amoo, T. B., Popoola, T., & Lucas, R. (2022) · Figshare · doi:10.6084/m9.figshare.19695957
Barnett, P., Steare, T., Dedat, Z., Pilling, S., McCrone, P., Knapp, M. R. J., Cooke, E., Lamirel, D., Dawson, S., Goldblatt, P., et al. (2022) · Figshare · doi:10.6084/m9.figshare.19681799
Barnett, P., Steare, T., Dedat, Z., Pilling, S., McCrone, P., Knapp, M. R. J., Cooke, E., Lamirel, D., Dawson, S., Goldblatt, P., et al. (2022) · Figshare · doi:10.6084/m9.figshare.19681790
Developing end-of-life care at a Portuguese nursing home through participatory action research
Galvão, C. (2022) · Lancaster EPrints (Lancaster University) · doi:10.17635/lancaster/thesis/1744
Kock, J. H. D., Latham, H. A., Leslie, S. J., Grindle, M., Munoz, S.-A., Ellis, L., Polson, R., & O’Malley, C. (2021) · Figshare · doi:10.6084/m9.figshare.13550821
Barnett, P., Steare, T., Dedat, Z., Pilling, S., McCrone, P., Knapp, M. R. J., Cooke, E., Lamirel, D., Dawson, S., Goldblatt, P., et al. (2022) · Figshare · doi:10.6084/m9.figshare.19681802
Barnett, P., Steare, T., Dedat, Z., Pilling, S., McCrone, P., Knapp, M. R. J., Cooke, E., Lamirel, D., Dawson, S., Goldblatt, P., et al. (2022) · Figshare · doi:10.6084/m9.figshare.19681793
Barnett, P., Steare, T., Dedat, Z., Pilling, S., McCrone, P., Knapp, M. R. J., Cooke, E., Lamirel, D., Dawson, S., Goldblatt, P., et al. (2022) · Figshare · doi:10.6084/m9.figshare.19681805
Barnett, P., Steare, T., Dedat, Z., Pilling, S., McCrone, P., Knapp, M. R. J., Cooke, E., Lamirel, D., Dawson, S., Goldblatt, P., et al. (2022) · Figshare · doi:10.6084/m9.figshare.19681808
Barnett, P., Steare, T., Dedat, Z., Pilling, S., McCrone, P., Knapp, M. R. J., Cooke, E., Lamirel, D., Dawson, S., Goldblatt, P., et al. (2022) · Figshare · doi:10.6084/m9.figshare.19681787
Barnett, P., Steare, T., Dedat, Z., Pilling, S., McCrone, P., Knapp, M. R. J., Cooke, E., Lamirel, D., Dawson, S., Goldblatt, P., et al. (2022) · Figshare · doi:10.6084/m9.figshare.19681796
Hicks, S., Davidson, M., Efstathiou, N., & Guo, P. (2022) · Figshare · doi:10.6084/m9.figshare.21622029
Leino, H. (2021) · Työväentutkimus Vuosikirja
Occupational well-being of nurse educators – development and evaluation of a digital intervention
Rinne, J. (2024) · UTUPub (University of Turku) · Cited by 1
Shoesmith, G. (2024) · Open Access at Essex (University of Essex)
An exploration of religion and desistance from crime
Saunders, N. (2024) · University of Birmingham Institutional Research Archive (University of Birmingham)
Delusional beliefs: Exploring associated cognitive and emotion factors
Fitzpatrick, D. (2024) · White Rose eTheses Online (University of Leeds, The University of Sheffield, University of York)
Violence Risk in Forensic and Clinical Populations of Men and Women
Gunenc, C. (2024) · UEA Digital Repository (University of East Anglia)
Weston, G. (2025) · Repository@Hull (Worktribe) (University of Hull) · Cited by 1
Fahrizki, D. D., & Dwiriani, C. M. (2025) · Springer Link (Chiba Institute of Technology) · doi:10.1051/bioconf/202515302009/pdf
Agboomaname, O. J., & Ristell, R. V. (2025) · Journal of Diabetes & Metabolic Disorders · doi:10.1007/s40200-025-01794-3
Ezeuko, A. Y.-A., Nduka, S. O., Oli, A. N., Chukwujekwu, N. P., Rowaiye, A. B., Ezeuko, K. K.-S., & Ekwunife, O. I. (2026) · Perspectives in Clinical Research · doi:10.4103/picr.picr_280_25
“[3,26,27] This study adopted a consensus level of ≥70% for the inclusion of items as triggers, following the recommendations of various studies.”
Support Programmes for Parents of Children with Intellectual Disabilities: A Scoping Review
Marais, J. E., Mthembu, T. G., & Wegner, L. (2026) · Commonwealth Youth and Development · doi:10.25159/2663-6549/19609
Physical factors and motivation in learning Dollyo Chagi kicks: a narrative review
Hartono, P., Rifki, M. S., Bakhtiar, S., & Okilanda, A. (2025) · JPPI (Jurnal Penelitian Pendidikan Indonesia) · doi:10.29210/020256709
Development-centred neonatal care: validation study of the patient safety checklist
Malveira, A. C., Teles, M., Lusquinhos, L., Dias, A., Oliveira, & Lopes, M. (2026) · Frontiers in Pediatrics · doi:10.3389/fped.2026.1754878
Crawford, M., Terry, R., & Jameel, L. (2026) · Psychology and Psychotherapy Theory Research and Practice · doi:10.1111/papt.70080
Fernandez, M. T., Moase, J., & Sladeczek, I. E. (2026) · Review Journal of Autism and Developmental Disorders · doi:10.1007/s40489-026-00559-9
Huang, P., Abai, D. S. A., Latip, M., Xiao, H., Zhang, Q., Yang, L., & Abdullah, K. (2026) · Public Health · doi:10.1016/j.puhe.2026.106407
Leahy, K., & McCormack, O. (2026) · International Journal of Technology and Design Education · doi:10.1007/s10798-026-10108-4
Ibrahim, M., Kowal, M. R., Mihaljevic, A. L., Limani, P., Lodge, J. P. A., & Kron, P. (2026) · Surgical Endoscopy · doi:10.1007/s00464-026-13132-6
“Alongside the traditional approach to reporting data, a structured narrative synthesis was conducted in line with the Guidance on the Conduct of Narrative Synthesis in Systematic Reviews from the Economic and Social Research Council [11].”
Akosile, C. N. (2026) · International journal of research and scientific innovation · doi:10.51244/ijrsi.2026.1307000298
Factors Influencing Older People's Disaster Preparedness Capacity: A Systematic Review
Lee, S.-J., & Hur, Y. (2026) · International Journal of Older People Nursing · doi:10.1111/opn.70099
Biomechanical interaction between saddle, rider, and the horse's back
Deckers, I. (2024) · UWE Research Repository (UWE Bristol)
Gorish, B. M. T., Abdelmula, W. I. Y., & Xu, F. (2026) · Acta Biomaterialia · doi:10.1016/j.actbio.2026.09.008
Domestic sensor-based monitoring for supporting independent-living older adults: A scoping review
Murshida, A., Potter, S., & Hawley, M. (2026) · Journal of Ambient Intelligence and Smart Environments · doi:10.1177/18761364261498839
Amaral, M., Correia, B., Oliveira, M., dos Santos, B., Ferreira, R. M., & Gonçalves, R. S. (2026) · Healthcare · doi:10.3390/healthcare14203388
“Formal statistical heterogeneity and sensitivity analyses were not performed because quantitative pooling was not undertaken and most outcome categories contained too few comparable studies to support reliable analyses [41,83].”
Comprehensive Competence of Dental Hygienists: A Mixed‐Methods Systematic Review
Karjalainen, K. A., Lammi, A. L. N., Oikarinen, A., & Kuivila, H. (2026) · International Journal of Dental Hygiene · doi:10.1111/idh.70187
Metadata
- Authors
- Campbell, M., Katikireddi, S. V., Sowden, A., McKenzie, J. E., & Thomson, H.
- Affiliations
- MRC/CSO Social and Public Health Sciences Unit (GB)
- University of York (GB)
- Monash University (AU)
- Corresponding author
- Campbell, Mhairi
- Venue
- BMJ Open
- Publisher
- BMJ
- Published
- 2018-02
- Volume
- 8
- Issue
- 2
- Pages
- e020064
- ISSN
- 2044-6055 (ISSN-L)
- PMC
- PMC5855342
- OpenAlex
- W2789982275
- Cited by
- 142
- References
- 19
- Citations by year
- 2026: 14 · 2025: 27 · 2024: 25 · 2023: 11 · 2022: 27 · 2021: 14 · 2020: 15 · 2019: 8 · 2018: 1
- FWCI
- 7.23
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- Locations
- BMJ Open (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- White Rose Research Online (University of Leeds, The University of Sheffield, University of York) (acceptedVersion, cc-by)
- ENLIGHTEN (Jurnal Bimbingan dan Konseling Islam) (acceptedVersion, cc-by-sa)
- Monash University Research Portal (Monash University) (submittedVersion, cc-by)
- Funders
- Cochrane Strategic Methods Fund
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Sociology and Political Science › Delphi Technique in Research
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Clinical practice guidelines implementation
- Keywords
- reporting guidelines, systematic reviews, reporting checklist, Delphi consensus
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.