Testing Methodological Guidance on the Conduct of Narrative Synthesis in Systematic Reviews: Effectiveness of Interventions to Promote Smoke Alarm Ownership and Function
The DOI matches its Crossref record; 22 of the 22 references it cites were found in a registry or an index.
Abstract
The objective was to assess the impact of new guidance on the conduct of narrative synthesis in systematic reviews of effectiveness, by means of a blinded comparison of guidance-led narrative synthesis against a meta-analysis of the same study data.The conclusions of the two syntheses were broadly similar. However, differences between the approaches meant that conclusions about the impact of moderators of effect appeared stronger when derived from the meta-analysis, whereas implications for future research appeared more extensive when derived from the narrative synthesis. These findings emphasize that a rigorously conducted narrative synthesis can add meaning and value to the findings of meta-analysis.The guidance framework provided a useful vehicle for structuring a narrative synthesis and increasing transparency and rigour of the process.While there may be risks with overinterpretation of study data, the framework, tools and techniques described in the guidance appear to increase the transparency and reproducibility of narrative synthesis.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“The data were analyzed using a combination of grouping techniques and constant comparison methods to identify key findings [19].”
From Examining the impact of economic abuse on survivors of intimate partner violence: a scoping review (2022) · cites this paper as [19]
In the citing paper
“Because studies had great heterogeneity in measuring and reporting mental health outcomes, a narrative rather a meta-analytical approach of synthesising findings was deemed appropriate [32].”
From Exploring the Impact of COVID-19 on Mental Health Outcomes in Children and Adolescents: A Systematic Review (2020) · cites this paper as [32]
Citations
Cites
Assessing the quality of reports of randomized clinical trials: Is blinding necessary?
Jadad, A. R., Moore, R. A., Carroll, D., Jenkinson, C., Reynolds, D. J. M., Gavaghan, D. J., & McQuay, H. J. (1996) · Controlled Clinical Trials · doi:10.1016/0197-2456(95)00134-4 · Cited by 17,582 · cited 13×
“It is unclear from the synthesis of RCTs presented here how specifi c fi re-related safety education compares with general safety advice.”
MethodInfluentialStatistical Methods for Meta-Analysis
Wasserman, S., Hedges, L. V., & Olkin, I. (1988) · Journal of Educational Statistics · doi:10.2307/1164953 · Cited by 10,326
The Handbook of Research Synthesis.
Howel, D., Cooper, H., & Hedges, L. V. (1995) · Biometrics · doi:10.2307/2533295 · Cited by 3,642
Statistical Methods for Meta-Analysis
(1985) · Elsevier eBooks · doi:10.1016/c2009-0-03396-0 · Cited by 2,521
Synthesising qualitative and quantitative evidence: A review of possible methods
Dixon-Woods, M., Agarwal, S., Jones, D., Young, B., & Sutton, A. (2005) · Journal of Health Services Research & Policy · doi:10.1177/135581960501000110 · Cited by 2,132 · cited 1×
“There are a number of ways in which research evidence can be brought together to give an overall picture of current knowledge that can be used to inform policy and practice decisions (Dixon-Woods et al., 2005).”
BackgroundPopay, J., Roberts, H., Sowden, A., Petticrew, M., Arai, L., Rodgers, M., Britten, N., Roen, K., & Duffy, S. (2006) · Lancaster University · doi:10.13140/2.1.1018.4643 · Cited by 2,070
Self-citationThe Handbook of Research Synthesis.
Light, R. J., Cooper, H., & Hedges, L. V. (1994) · Journal of the American Statistical Association · doi:10.2307/2291021 · Cited by 1,868
Campbell, R., Pound, P., Pope, C., Britten, N., Pill, R., Morgan, M., & Donovan, J. (2003) · Social Science & Medicine · doi:10.1016/s0277-9536(02)00064-3 · Cited by 921 · cited 1×
“Because of the perceived lack of prior knowledge and expertise required, reviewers may select a ‘simple’ narrative synthesis over potentially applicable but more complex approaches such as meta-ethnography (Campbell et al., 2003) or Bayesian methods (Roberts et al., 2002).”
BackgroundSelf-citationRealist synthesis - an introduction
Pawson, R., Greenhalgh, T., Harvey, G., & Walshe, K. (2004) · UCL Discovery (University College London) · Cited by 906
Analysing and Presenting Results
Macaskill, P., Gatsonis, C., Deeks, J. J., Harbord, R., & Takwoingi, Y. (2010) · Bristol Research (University of Bristol) · Cited by 337
Arai, L., Britten, N., Popay, J., Roberts, H., Petticrew, M., Rodgers, M., & Sowden, A. (2007) · Evidence & Policy: A Journal of Research, Debate and Practice · doi:10.1332/174426407781738029 · Cited by 177 · cited 2×
“We were funded to develop guidance that would help improve the quality of narrative synthesis in systematic reviews (see Box 1) (Arai et al., 2007; Popay et al., 2006).”
MethodSelf-citationRoberts, K. A., Dixon-Woods, M., Fitzpatrick, R., Abrams, K. R., & Jones, D. R. (2002) · The Lancet · doi:10.1016/s0140-6736(02)11560-1 · Cited by 159 · cited 2×
“, 2003) or Bayesian methods (Roberts et al., 2002).”
BackgroundGuidance on the Conduct of Narrative Synthesis in Systematic Reviews Final Report
Jrh, P., Roberts, H. M., Aj, S., Petticrew, M. P., Britten, N., Arai, L., Roen, K., & Rodgers, M. E. (2005) · Cited by 152
Self-citationClamp, M., & Kendrick, D. (1998) · BMJ · doi:10.1136/bmj.316.7144.1576 · Cited by 127
The Effectiveness of a Home Visit to Prevent Childhood Injury
King, W. J., Klassen, T. P., LeBlanc, J., Bernard-Bonnin, A.-C., Robitaille, Y., Pham, B., Coyle, D., Tenenbein, M., & Pless, I. B. (2001) · Pediatrics · doi:10.1542/peds.108.2.382 · Cited by 125 · cited 1×
“Another subgroup meta-analysis indicated that the removal of the one study in which a research assistant delivered the intervention (King et al., 2001) resulted in a stronger positive effect of intervention on three of the reported outcomes.”
Preventing injuries in children: cluster randomised controlled trial in primary care
Kendrick, D., Marsh, P., Fielding, K., & Miller, P. (1999) · BMJ · doi:10.1136/bmj.318.7189.980 · Cited by 106 · cited 2×
“The authors state that these subgroup analyses were based on few trials and were heavily infl uenced by a single trial (Kendrick et al., 1999).”
BackgroundResultEvaluation of Group Well-Child Care for Improving Burn Prevention Practices in the Home
Thomas, K. A., HASSANEIN, R. S., & Christophersen, E. R. (1984) · PEDIATRICS · doi:10.1542/peds.74.5.879 · Cited by 101
Safety Education in a Pediatric Primary Care Setting
Kelly, B., Sein, C., & McCarthy, P. L. (1987) · Pediatrics · doi:10.1542/peds.79.5.818 · Cited by 97 · cited 2×
“Smoke alarm ownership in the control groups of each study was generally quite high, with one clear exception (Kelly et al., 1987: 11 percent).”
BackgroundResultInterventions for promoting smoke alarm ownership and function
DiGuiseppi, C., Goss, C. W., & Higgins, J. P. (2001) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd002246 · Cited by 81
A Randomized Single-Blind Evaluation of a Discharge Teaching Book for Pediatric Patients With Burns
Jenkins, H. M. L., Blank, V., Miller, K., Turner, J., & Stanwick, R. S. (1996) · Journal of Burn Care & Rehabilitation · doi:10.1097/00004630-199601000-00012 · Cited by 43
Meta‐Narrative Mapping: A New Approach to the Systematic Review of Complex Evidence
Greenhalgh, T. (2004) · Wiley eBooks · doi:10.1002/9780470755167.ch21 · Cited by 40 · cited 1×
“This should not be confused with approaches such as meta-narrative mapping which specifi cally aims to construct a structurally and thematically coherent story to describe a complex body of evidence (Greenhalgh, 2004).”
BackgroundGet Fired Up: A Model for Developing In-School 4-H Programs
Davis, J. W. (1987) · TigerPrints (Clemson University) · Cited by 3
Research works printed but not linked: 4
- Barone, V.J., 1988, `An Analysis of Well-Child Parenting Classes: The Extent of Parent Compliance with Health Care Recommendations to Decrease Potential Injury of their Toddlers', dissertation
- Jones, D., 2003, `Bayesian Synthesis of Qualitative and Quantitative Evidence: Overview and Example'
- Mathews, J.R., 1988, `An Analysis of Dangerous Behaviour in Toddlers', dissertation
- Williams, G.E., 1988, `An Analysis of Prenatal Education Classes: An Early Start to Injury Prevention', dissertation
Cited by
Systematic reviews: CRD's guidance for undertaking reviews in health care
Tacconelli, E. (2010) · The Lancet Infectious Diseases · doi:10.1016/s1473-3099(10)70065-7 · Cited by 3,245
Synthesizing information systems knowledge: A typology of literature reviews
Paré, G., Trudel, M.-C., Jaana, M., & Kitsiou, S. (2015) · Information & Management · doi:10.1016/j.im.2014.08.008 · Cited by 1,987
Online communication, social media and adolescent wellbeing: A systematic narrative review
Best, P., Manktelow, R., & Taylor, B. (2014) · Children and Youth Services Review · doi:10.1016/j.childyouth.2014.03.001 · Cited by 1,067
“In most cases the final product of such reviews is the presentation of a statistical (quantitative) or narrative (qualitative) summary of findings (Rodgers et al, 2009).”
Regmi, K., & Jones, L. (2020) · BMC Medical Education · doi:10.1186/s12909-020-02007-6 · Cited by 751
Muka, T., Glisic, M., Milic, J., Verhoog, S., Bohlius, J., Bramer, W., Chowdhury, R., & Franco, O. H. (2020) · European Journal of Epidemiology · doi:10.1007/s10654-019-00576-5 · Cited by 660
Metadata
- Authors
- Rodgers, M., Sowden, A., Petticrew, M., Arai, L., Roberts, H., Britten, N., & Popay, J.
- Affiliations
- University of York (GB)
- London School of Hygiene & Tropical Medicine (GB)
- The Open University (GB)
- Institute of Education, UK,
- University of Exeter (GB)
- Lancaster University (GB)
- Corresponding author
- Rodgers, Mark
- Sowden, Amanda
- Petticrew, Mark
- Arai, Lisa
- Roberts, Helen
- Britten, Nicky
- Popay, Jennie
- Venue
- Evaluation
- Publisher
- SAGE Publications
- Published
- 2009-01
- Volume
- 15
- Issue
- 1
- Pages
- 49-73
- ISSN
- 1356-3890 (ISSN-L)
- 1461-7153
- OpenAlex
- W2094738656
- Cited by
- 561
- References
- 22
- Citations by year
- 2026: 72 · 2025: 80 · 2024: 58 · 2023: 47 · 2022: 32 · 2021: 29 · 2020: 48 · 2019: 24 · 2018: 21 · 2017: 25 · 2016: 19 · 2015: 18 · 2014: 32 · 2013: 22 · 2012: 18
- FWCI
- 0.81
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- journals.sagepub.com/page/policies/text-and-data-mining-license (tdm)
- Locations
- Evaluation (publishedVersion)
- Teesside University Research Portal (Teesside University) (submittedVersion)
- LSHTM Research Online (London School of Hygiene and Tropical Medicine) (submittedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Public Administration › Evaluation and Performance Assessment
- Social Sciences › Sociology and Political Science › Delphi Technique in Research
- Keywords
- narrative synthesis, systematic reviews, meta-analysis, methodological recommendations, transparency, reproducibility, overinterpretation, effectiveness research
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.