The DOI matches its Crossref record; 95 of the 97 references it cites were found in a registry or an index.
Development of methodological guidance, publication standards and training materials for realist and meta-narrative reviews: the RAMESES (Realist And Meta-narrative Evidence Syntheses – Evolving Standards) project
Open accessAbstract
Background: There is growing interest in theory-driven, qualitative and mixed-method approaches to systematic review, such as realist and meta-narrative review. These approaches offer the potential to expand the knowledge base in policy-relevant areas. However, the quality of such reviews can be difficult to assess. Objectives: The aim of this project was to produce methodological guidance, publication standards and training resources for those seeking to undertake realist and/or meta-narrative reviews. Methods/design: We (1) collated and summarised existing literature on the principles of good practice in realist and meta-narrative systematic reviews; (2) considered the extent to which these principles had been followed by published and in-progress reviews, thereby identifying how rigour may have been lost and how existing methods could be improved; (3) used an online Delphi method with an interdisciplinary panel of experts from academia and policy, to produce a draft set of methodological steps and publication standards; (4) produced training materials with learning objectives linked to these steps; (5) refined these standards and training materials prospectively on real reviews in progress, capturing methodological and other challenges as they arose; (6) synthesised expert input, evidence review and real-time problem analysis into more definitive guidance and standards; and (7) disseminated outputs to audiences in academia and policy. Results: An important element of this study was the establishment of an e-mail mailing list to bring together researches in the field ( www.jiscmail.ac.uk/RAMESES ). Our literature review identified 35 and nine realist and meta-narrative reviews respectively. Analysis and discussion within the project team produced a summary of the published literature, and common questions and challenges into briefing materials for the Delphi panel, comprising 37 and 33 members (for realist and meta-narrative reviews respectively). Within three rounds this panel had reached a consensus on 19 (realist) and 20 (meta-narrative) key publication standards, with an overall response rate of 90% and 91% respectively. The Realist And Meta-narrative Evidence Syntheses – Evolving Standards (RAMESES) publication standards for realist syntheses and meta-narrative reviews were published in open-access journals and quickly became highly accessed. The RAMESES quality standards and training materials drew together the following sources of data: (1) personal expertise as researchers and trainers; (2) data from the Delphi panels; (3) feedback from participants at training sessions we ran; and (4) comments made on RAMESES mailing list. The quality standards and training materials are freely available online ( www.ramesesproject.org ). Discussion: The production of these standards and guidance drew on multiple sources of knowledge and expertise, and a high degree of a consensus was achieved despite ongoing debate among researchers about the overall place of these methodologies in the secondary research toolkit. As with all secondary research methods, guidance on quality assurance and uniform reporting is an important step towards improving quality and consistency of studies. We anticipate that as more reviews are undertaken, further refinement will be needed to the publication and quality standards and training materials. Limitations: The project’s outputs are not definitive and in the future updating and further development is likely to be needed. Conclusion: An initial set of publication standards, quality standards and training materials have been produced for researchers, users and funders of realist or meta-narrative reviews. As realist and meta-narrative reviews are relatively new approaches to evidence synthesis, methodological development is needed for both review approaches. Funding: The National Institute for Health Research Health Services and Delivery Research programme.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“The realist approach followed processes set out by Pawson et al. and the Realist and Meta-narrative Evidence Synthesis: Evolving Standards (RAMESES) [29, 32].”
From Do hospital-to-home transitions work for older adults with multiple long-term conditions including dementia? A realist review (2025) · cites this paper as [32]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Liberati, A., Altman, D. G., Tetzlaff, J., Mulrow, C., Gotzsche, P. C., Ioannidis, J. P. A., Clarke, M., Devereaux, P. J., Kleijnen, J., & Moher, D. (2009) · BMJ · doi:10.1136/bmj.b2700 · Cited by 17,648
Methods for the thematic synthesis of qualitative research in systematic reviews
Thomas, J., & Harden, A. (2008) · BMC Medical Research Methodology · doi:10.1186/1471-2288-8-45 · Cited by 9,312
Moher, D., Hopewell, S., Schulz, K. F., Montori, V., Gotzsche, P. C., Devereaux, P. J., Elbourne, D., Egger, M., & Altman, D. G. (2010) · BMJ · doi:10.1136/bmj.c869 · Cited by 8,883
Diffusion of Innovations in Service Organizations: Systematic Review and Recommendations
GREENHALGH, T., ROBERT, G., MACFARLANE, F., BATE, P., & KYRIAKIDOU, O. (2004) · The Milbank Quarterly · doi:10.1111/j.0887-378x.2004.00325.x · Cited by 7,448
Self-citationResearch guidelines for the Delphi survey technique
Hasson, F. (2000) · Journal of Advanced Nursing · doi:10.1046/j.1365-2648.2000.01567.x · Cited by 5,951
The Delphi Technique: Making Sense of Consensus
Hsu, C.-C., & Sandford, B. A. (2007) · University of Massachusetts Amherst · doi:10.7275/pdz9-th90 · Cited by 4,500
Realist review - a new method of systematic review designed for complex policy interventions
Pawson, R., Greenhalgh, T., Harvey, G., & Walshe, K. (2005) · Journal of Health Services Research & Policy · doi:10.1258/1355819054308530 · Cited by 3,190 · cited 3×
“The quality standards and training materials drew on the following sources of data: (1) personal expertise as researchers and trainers; (2) data from the Delphi panels; (3) feedback from participants at training sessions we ran; and (4) comments made on RAMESES mailing list.”
MethodSelf-citationSynthesising qualitative and quantitative evidence: A review of possible methods
Dixon-Woods, M., Agarwal, S., Jones, D., Young, B., & Sutton, A. (2005) · Journal of Health Services Research & Policy · doi:10.1177/135581960501000110 · Cited by 2,132 · cited 4×
“Two of our initial questions related to these issues: (5) What is the link between realist and meta-narrative review and the policy-making process? How have published reviews been commissioned or sponsored? How have policy-makers been involved in shaping the review? How have they been involved in disseminating and app…”
BackgroundMethodInfluentialGreenhalgh, T., & Peacock, R. (2005) · BMJ · doi:10.1136/bmj.38636.593461.68 · Cited by 2,045
Self-citationMethods for the synthesis of qualitative research: a critical review
Barnett-Page, E., & Thomas, J. (2009) · BMC Medical Research Methodology · doi:10.1186/1471-2288-9-59 · Cited by 2,042
Research works printed but not linked: 3
- Weiss, 1995, New Approaches to Evaluating Community Initiatives: Concepts, Methods, and Contexts, 65
- Meyer, 2008, Independent Learning Literature Review (Research Report DCSF-RR051)
- McLean, 2010, Behavioral caregiving for adults with traumatic brain injury living in nursing homes: developing a practice model, J Theory Constr Test, 14, 17
Cited by
Munn, Z., Stern, C., Aromataris, E., Lockwood, C., & Jordan, Z. (2018) · BMC Medical Research Methodology · doi:10.1186/s12874-017-0468-4 · Cited by 1,164
RAMESES II reporting standards for realist evaluations
Wong, G., Westhorp, G., Manzano, A., Greenhalgh, J., Jagosh, J., & Greenhalgh, T. (2016) · BMC Medicine · doi:10.1186/s12916-016-0643-1 · Cited by 880
“In other words the data gathered needs to contain information that [20, 48]: (1)May be used to develop, support, refute or refine the assignment of the conceptual label of C, M or O”
MethodSelf-citationPollock, A., & Berge, E. (2018) · International Journal of Stroke · doi:10.1177/1747493017743796 · Cited by 530
“Realist And Metanarrative Evidence Syntheses – Evolving Standards (RAMESES) publication standards (126)”
BackgroundOrganizational Health Literacy: Review of Theories, Frameworks, Guides, and Implementation Issues
Farmanova, E., Bonneville, L., & Bouchard, L. (2018) · INQUIRY: The Journal of Health Care Organization, Provision, and Financing · doi:10.1177/0046958018757848 · Cited by 264
“We used an adaptation of the metanarrative review as a means for synthesizing a range of literature.(34) This review provides unique insights into the extent and scope of research and implementation of OHL and discusses recommendations to improve the organization, quality, and delivery of healthcare.”
MethodDada, S., Dalkin, S., Gilmore, B., Hunter, R., & Mukumbang, F. C. (2023) · Research Synthesis Methods · doi:10.1002/jrsm.1630 · Cited by 212
“Within prominent realist review methods papers and guidelines, terms such as credible, plausible, believable, trustworthy and coherent have been used individually or in combination to describe rigour.(16,18,19,21) This difficulty of defining and applying rigour is compounded when using data from different research par…”
BackgroundMethodBunn, F., Goodman, C., Russell, B., Wilson, P., Manthorpe, J., Rait, G., Hodkinson, I., & Durand, M.-A. (2018) · BMC Geriatrics · doi:10.1186/s12877-018-0853-9 · Cited by 207
“Theories rather than ‘programmes’ are the basic unit of analysis.(82) METHODS”
BackgroundProtocol—the RAMESES II study: developing guidance and reporting standards for realist evaluation
Greenhalgh, T., Wong, G., Jagosh, J., Greenhalgh, J., Manzano, A., Westhorp, G., & Pawson, R. (2015) · BMJ Open · doi:10.1136/bmjopen-2015-008567 · Cited by 197
Self-citationWong, G., Westhorp, G., Greenhalgh, J., Manzano, A., Jagosh, J., & Greenhalgh, T. (2017) · Health Services and Delivery Research · doi:10.3310/hsdr05280 · Cited by 196
“used in another project (the RAMESES project), which developed methodological guidance, reporting standards and training materials for realist and meta-narrative reviews.(14) We have previously published a protocol paper that outlined the methods we intended to use in this project.”
MethodSelf-citationHarris, J. L., Booth, A., Cargo, M., Hannes, K., Harden, A., Flemming, K., Garside, R., Pantoja, T., Thomas, J., & Noyes, J. (2018) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2017.10.023 · Cited by 165
Chinn, D., & Homeyard, C. (2016) · Health Expectations · doi:10.1111/hex.12520 · Cited by 167
Papoutsi, C., Mattick, K. L., Pearson, M., Brennan, N., Briscoe, S., & Wong, G. (2017) · Journal of Antimicrobial Chemotherapy · doi:10.1093/jac/dkx194 · Cited by 148
Self-citationPaparini, S., Papoutsi, C., Murdoch, J., Green, J., Petticrew, M., Greenhalgh, T., & Shaw, S. E. (2021) · BMC Medical Research Methodology · doi:10.1186/s12874-021-01418-3 · Cited by 131
Self-citationHunter, R. L., Gorely, T., Beattie, M., & Harris, K. (2022) · International Review of Sport and Exercise Psychology · doi:10.1080/1750984x.2021.1969674 · Cited by 143
“A realist review is a type of systematic literature review which is characterised by its explanatory focus (Wong et al., 2014).”
BackgroundMethodInfluentialColes, E., Wells, M., Maxwell, M., Harris, F., Anderson, J. M., Gray, N. M., Milner, G., & MacGillivray, S. A. (2017) · Systematic Reviews · doi:10.1186/s13643-017-0566-8 · Cited by 138
Reeve, J. L., Maden, M., Hill, R. A., Turk, A., Mahtani, K. R., Wong, G., Lasserson, D. S., Krska, J., Mangin, D. A., Byng, R., et al. (2022) · Health Technology Assessment · doi:10.3310/aafo2475 · Cited by 133
Self-citationCarrieri, D., Briscoe, S., Jackson, M., Mattick, K. L., Papoutsi, C., Pearson, M., & Wong, G. (2018) · BMJ Open · doi:10.1136/bmjopen-2017-021273 · Cited by 105
“Evidence Syntheses—Evolving Standards quality and publication standards for realist reviews.(50) 51”
BackgroundSelf-citationRapid realist review of the role of community pharmacy in the public health response to COVID-19
Maidment, I. D., Young, E. L., MacPhee, M., Booth, A., Zaman, H., Breen, J., Hilton, A., Kelly, T., & Wong, G. (2021) · BMJ Open · doi:10.1136/bmjopen-2021-050043 · Cited by 86
Self-citationUnderstanding the role of the paramedic in primary care: a realist review
Eaton, G., Wong, G., Tierney, S., Roberts, N. W., Williams, V., & Mahtani, K. R. (2021) · BMC Medicine · doi:10.1186/s12916-021-02019-z · Cited by 100
“transparently, in accordance with the RAMSES quality standards [44].”
ResultSelf-citationKastner, M., Hayden, L., Wong, G., Lai, Y., Makarski, J., Treister, V., Chan, J., Lee, J. H., Ivers, N. M., Holroyd-Leduc, J. M., et al. (2019) · BMJ Open · doi:10.1136/bmjopen-2018-025009 · Cited by 95
Self-citationBergerum, C., Thor, J., Josefsson, K., & Wolmesjö, M. (2019) · Health Expectations · doi:10.1111/hex.12900 · Cited by 103
Klatte, I. S., Lyons, R., Davies, K., Harding, S. A., Marshall, J., McKean, C., & Roulstone, S. (2020) · International Journal of Language & Communication Disorders · doi:10.1111/1460-6984.12538 · Cited by 103
“Realist evaluation was originally developed by Pawson and Tilley (1997) in the 1990s to explore systematically the link between interventions and outcomes, summarized as ‘what works, how, for whom, in what circumstances and to what extent’ (Wong et al. 2014: 1).”
MethodDuddy, C., & Wong, G. (2022) · BMJ Quality & Safety · doi:10.1136/bmjqs-2022-015236 · Cited by 98
Self-citationMacDonald, M., Pauly, B., Wong, G., Schick‐Makaroff, K., van Roode, T., Strosher, H., Kothari, A. R., Valaitis, R. K., Manson, H. E., O’Briain, W., et al. (2016) · Systematic Reviews · doi:10.1186/s13643-016-0229-1 · Cited by 84
“The notion of ‘mechanism’ is thus central in realism for explaining the relationship between the social world (context) and human actions or behaviour (outcomes) [27].”
Self-citationBalogun, O. O., Dagvadorj, A., Yourkavitch, J., Lopes, K. D. S., Suto, M., Takemoto, Y., Mori, R., Rayco‐Solon, P., & Ota, E. (2017) · Breastfeeding Medicine · doi:10.1089/bfm.2017.0040 · Cited by 74
Development of a program theory for shared decision-making: a realist synthesis
Waldron, T., Carr, T., McMullen, L. M., Westhorp, G., Duncan, V., Neufeld, S.-M., Bandura, L.-A., & Groot, G. (2020) · BMC Health Services Research · doi:10.1186/s12913-019-4649-1 · Cited by 97
“Meta-narrative Evidence Syntheses: Evolving Standards (RAMESES) realist training guidelines [50, 53, 54].”
BackgroundSelf-citationInforming NHS policy in ‘digital-first primary care’: a rapid evidence synthesis
Rodgers, M., Raine, G., Thomas, S., Harden, M., & Eastwood, A. J. (2019) · Health Services and Delivery Research · doi:10.3310/hsdr07410 · Cited by 69
Carrieri, D., Mattick, K. L., Pearson, M., Papoutsi, C., Briscoe, S., Wong, G., & Jackson, M. (2020) · BMC Medicine · doi:10.1186/s12916-020-01532-x · Cited by 79
Self-citationPerson‐centred care in primary care: What works for whom, how and in what circumstances?
Ahmed, A., van den Muijsenbergh, M. E. T. C., & Vrijhoef, H. J. (2022) · Health & Social Care in the Community · doi:10.1111/hsc.13913 · Cited by 113
“(Wong et al., 2014).”
Brown, A. D., Lafreniere, K., Freedman, D., Nidumolu, A., Mancuso, M., Hecker, K. G., & Kassam, A. (2020) · BMJ Quality & Safety · doi:10.1136/bmjqs-2020-010887 · Cited by 54
Abrams, R., Park, S. E., Wong, G., Rastogi, J., Boylan, A. R., Tierney, S., Petrova, M., Dawson, S., & Roberts, N. W. (2020) · Research Synthesis Methods · doi:10.1002/jrsm.1459 · Cited by 62
“The Realist And MEta-narrative Evidence Syntheses: Evolving Standards (RAMESES) were published in 2014; despite not setting any standards for involving contributors, they acted as an international standard for undertaking realist reviews more broadly.(18) The purpose of this review was to (1) describe the ways in whic…”
MethodSelf-citationCalderón‐Larrañaga, S., Milner, Y., Clinch, M., Greenhalgh, T., & Finer, S. (2021) · BJGP Open · doi:10.3399/bjgpo.2021.0017 · Cited by 57
“The framework has been developed systematically, following rigorous methodological guidance for realist reviews as described in the RAMESES quality standards.21,22 The iterative nature of the realist approach has enhanced the scope and practical relevance of the framework, by incorporating relevant studies that would…”
BackgroundMethodInfluentialSelf-citationOptimising the delivery of remediation programmes for doctors: A realist review
Price, T., Wong, G., Withers, L., Wanner, A., Cleland, J., Gale, T. C. E., Prescott‐Clements, L., Archer, J. C., Bryce, M., & Brennan, N. (2021) · Medical Education · doi:10.1111/medu.14528 · Cited by 58
“16 The review was conducted by a ten person review team, which com - prised six active medical education researchers, a realist method expert, an information specialist, a former lead intervention and as - sessment advisor for the Patient Performance and Advisory Service (formerly the National…”
BackgroundSelf-citationMorton, T. A., Wong, G., Atkinson, T. J., & Brooker, D. (2021) · BMJ Open · doi:10.1136/bmjopen-2020-047789 · Cited by 54
Self-citationSiersbaek, R., Ford, J. A., Burke, S., Cheallaigh, C. N., & Thomas, S. (2021) · BMJ Open · doi:10.1136/bmjopen-2020-043091 · Cited by 47
“In this review we have used the RAMESES quality standards (70) to rigorously and systematically review the literature included here.”
MethodVincent, R., Adhikari, B., Duddy, C., Richardson, E. Z. L., Wong, G., Lavery, J. V., Molyneux, S. C., & The REAL team: Mary Chambers, P. Y. C. (2022) · Wellcome Open Research · doi:10.12688/wellcomeopenres.17192.1 · Cited by 45
Self-citationRowe, A., Titterington, J., Holmes, J., Henry, L. A., & Taggart, L. (2019) · Developmental Review · doi:10.1016/j.dr.2019.02.001 · Cited by 56
“Thus, the questions posed in this review are consistent with a realist research perspective (Fletcher et al., 2016; Wong, Greenhalgh, Westhorp, & Pawson, 2014).”
Responsible and human centric AI-based insurance advisors
Pisoni, G., & Díaz-Rodríguez, N. (2023) · Information Processing & Management · doi:10.1016/j.ipm.2023.103273 · Cited by 42
A realist review of family-based interventions for children of substance abusing parents
Usher, A. M., McShane, K., & Dwyer, C. L. (2015) · Systematic Reviews · doi:10.1186/s13643-015-0158-4 · Cited by 48
A realist review of community engagement with health research
Adhikari, B., Vincent, R., Wong, G., Duddy, C., Richardson, E. Z. L., Lavery, J. V., & Molyneux, S. C. (2019) · Wellcome Open Research · doi:10.12688/wellcomeopenres.15298.2 · Cited by 59
Self-citationCardiff, S., Gershuni, O., & Brekelmans, A. (2023) · Journal of Clinical Nursing · doi:10.1111/jocn.16813 · Cited by 34
Provencher, V., D’Amours, M., Menear, M., Obradović, N., Veillette, N., Sirois, M., & Kergoat, M. (2021) · BMC Geriatrics · doi:10.1186/s12877-020-01980-3 · Cited by 43
“Abbreviations ICMO: Intervention-Context-Mechanism-Outcome; ISBAR: Identify, Situation, Background, Assessment and Recommendation; RAMESES: Realist And Metanarrative Evidence Syntheses: Evolving Standards; RCT: Randomized Control Trials; STEPS: Safe and Successful Transition of Elderly Patients Study Acknowledgements…”
MethodInfluentialMusa, M. K., Akdur, G., Brand, S. L., Killett, A., Spilsbury, K., Peryer, G., Burton, J. K., Gordon, A. L., Hanratty, B., Towers, A., et al. (2022) · BMC Geriatrics · doi:10.1186/s12877-021-02705-w · Cited by 40
“This review draws on practical ‘how-to’ guidance [34, 37], and follows publication standards (Realist And”
BackgroundSmith, L., POTTS, H. W. W., Amlôt, R., Fear, N. T., Michie, S., & Rubin, G. J. (2022) · BMJ Open · doi:10.1136/bmjopen-2021-058060 · Cited by 30
Crosbie, B., Ferguson, M., Wong, G., Walker, D., Vanhegan, S., & Dening, T. (2019) · BMC Medicine · doi:10.1186/s12916-019-1286-9 · Cited by 32
“When undertaking our RS, we followed the RAMESES quality and publication standards [35].”
MethodSelf-citationAbrams, R., Wong, G., Mahtani, K. R., Tierney, S., Boylan, A. R., Roberts, N. W., & Park, S. E. (2020) · British Journal of General Practice · doi:10.3399/bjgp20x710153 · Cited by 34
Self-citationTurk, A., Wong, G., Mahtani, K. R., Maden, M., Hill, R. A., Ranson, E., Wallace, E., Krska, J., Mangin, D. A., Byng, R., et al. (2022) · BMC Medicine · doi:10.1186/s12916-022-02475-1 · Cited by 43
“Step 1: Clarifying the scope Realist reviews begin with the development of an initial “draft” theory of how an intervention is understood to work—a programme theory [21].”
BackgroundMethodSelf-citationUnderstanding the causes of missingness in primary care: a realist review
Lindsay, C., Baruffati, D., Mackenzie, M., Ellis, D. A., Major, M., O’Donnell, C. A., Simpson, S. A., Williamson, A. E., & Wong, G. (2024) · BMC Medicine · doi:10.1186/s12916-024-03456-2 · Cited by 36
“The review follows the Realist And MEta-narrative Evidence Synthe-ses: Evolving Standards (RAMESES) project standards for realist synthesis [25, 26].”
MethodSelf-citationGoodman, C. M., Norton, C. S., Buswell, M., Russell, B., Harari, D., Harwood, R., Roe, B. H., Rycroft‐Malone, J., Drennan, V. M., Fader, M. J., et al. (2017) · Health Technology Assessment · doi:10.3310/hta21420 · Cited by 36
Young carers, mental health and psychosocial wellbeing: A realist synthesis
Janes, E., Forrester, D., Reed, H., & Meléndez‐Torres, G. J. (2021) · Child Care Health and Development · doi:10.1111/cch.12924 · Cited by 37
“The procedure used to identify, screen and analyse papers to develop the model follows the RAMESES realist protocols (Wong et al., 2014).”
MethodCradock-Henry, N. A., Kirk, N., Ricart, S., Diprose, G., & Kannemeyer, R. L. (2023) · Environmental Research Letters · doi:10.1088/1748-9326/ace0ce · Cited by 36
Kenny, A. J., Dickson‐Swift, V., Gussy, M. G., Kidd, S., Cox, D., Masood, M., Azul, D., Chan, C. K. Y., Christian, B., Theobold, J., et al. (2020) · International Journal of Mental Health Systems · doi:10.1186/s13033-020-00357-8 · Cited by 41
Participatory ergonomics: What works for whom and why? A realist review
Hansen, K., Hasle, P., Caroly, S., Reinhold, K., Järvis, M., Herrig, A. O., Heiberg, B. D., Søgaard, K., Punnett, L., & Stochkendahl, M. J. (2023) · Ergonomics · doi:10.1080/00140139.2023.2202842 · Cited by 51
Boyko, J. A., Riley, B., Willis, C. D., Stockton, L. D., Zummach, D., Kerner, J. F., Robinson, K. L., & Chia, M. C. (2018) · Health Research Policy and Systems · doi:10.1186/s12961-018-0374-1 · Cited by 31
Roberts, J. L., Din, N. U., Williams, M. D., Hawkes, C., Charles, J. M., Hoare, Z., Morrison, V. L., Alexander, S., LEMMEY, A. B., Sackley, C. M., et al. (2017) · BMJ Open · doi:10.1136/bmjopen-2016-014362 · Cited by 25
Meta-synthesis of Qualitative Research
Dawson, A. J. (2019) · Springer eBooks · doi:10.1007/978-981-10-5251-4_112 · Cited by 33
Hersh, D., Azul, D., Carroll, C., Lyons, R., McMenamin, R., & Skeat, J. (2022) · International Journal of Speech-Language Pathology · doi:10.1080/17549507.2022.2029942 · Cited by 34
Maniaci, A., Saibene, A. M., Calvo‐Henríquez, C., Vaira, L. A., Radulesco, T., Michel, J., Chiesa-Estomba, C. M., Sowerby, L. J., Lobo, D., Mayo‐Yáñez, M., et al. (2024) · European Archives of Oto-Rhino-Laryngology · doi:10.1007/s00405-024-08464-9 · Cited by 30
“Data were extracted independently by two authors, with disagreement resolved by consensus and synthesized using a narrative approach due to the heterogeneity of the studies [23].”
Williams, N. H., Roberts, J. L., Din, N., Charles, J. M., Totton, N., Williams, M. D., Mawdesley, K., Hawkes, C., Morrison, V. L., LEMMEY, A. B., et al. (2017) · Health Technology Assessment · doi:10.3310/hta21440 · Cited by 32
Abayneh, S., Lempp, H. K., Manthorpe, J., & Hanlon, C. (2018) · International Journal of Mental Health Systems · doi:10.1186/s13033-018-0220-4 · Cited by 27
“Compared to conventional systematic reviews, in realist reviews it is a challenge to reproduce the review because of the utilisation of a mixture of evidence and processes [36, 55].”
BackgroundMethodA Tailored Approach: A model for literature searching in complex systematic reviews
Cooper, C., Booth, A., Husk, K., Lovell, R., Frost, J., Schauberger, U., Britten, N., & Garside, R. (2022) · Journal of Information Science · doi:10.1177/01655515221114452 · Cited by 26
Abrams, R., Wong, G., Mahtani, K. R., Tierney, S., Boylan, A. R., Roberts, N. W., & Park, S. E. (2018) · BMJ Open · doi:10.1136/bmjopen-2018-024876 · Cited by 22
“This review will follow the Realist and Meta-Review Evidence Synthesis: Evolving Standards guidelines on quality and reporting.(32)”
MethodSelf-citationMaking theory from knowledge syntheses useful for public health
Wong, G. (2018) · International Journal of Public Health · doi:10.1007/s00038-018-1098-2 · Cited by 23
“making sense of complex interventions where context is thought to influence outcomes and answer questions that ask some or all of what works, for whom, in what contexts, to what extent, how and why? Reporting and quality standards, and training materials exist for realist reviews (Wong et al. 2014) and realist evaluat…”
BackgroundSelf-citationDixon, S., Duddy, C., Harrison, G., Papoutsi, C., Ziébland, S., & Griffiths, F. E. (2021) · BMJ Open · doi:10.1136/bmjopen-2020-039809 · Cited by 26
Preventing Childhood Obesity in Primary Schools: A Realist Review from UK Perspective
Ijaz, S., Nobles, J., Johnson, L., Moore, T. H., Savović, J., & Jago, R. (2021) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph182413395 · Cited by 22
“Patient and Public Involvement: We sought external stakeholder opinion [13,14]—via an online consultation—to facilitate our understanding of the UK primary school contexts, and what stakeholders (school staff, management and organisations that work with primary schools) consider important for our review’s question.”
BackgroundZhao, J., Jull, J., Finderup, J., Smith, M., Kienlin, S. M., Rahn, A. C., Dunn, S., Aoki, Y., Brown, L., Harvey, G., et al. (2022) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-022-02007-0 · Cited by 23
“The review involved six iterative steps [15, 16] as follows (see Fig.”
BackgroundMethodMukumbang, F. C., & Wong, G. (2025) · Qualitative Health Research · doi:10.1177/10497323251316401 · Cited by 33
Self-citationRemediation programmes for practising doctors to restore patient safety: the RESTORE realist review
Price, T., Brennan, N., Wong, G., Withers, L., Cleland, J., Wanner, A., Gale, T. C. E., Prescott‐Clements, L., Archer, J. C., & Bryce, M. (2021) · Health Services and Delivery Research · doi:10.3310/hsdr09110 · Cited by 23
Self-citationStasiłowicz-Krzemień, A., Nogalska, W., Maszewska, Z., Maleszka, M., Dobroń, M., Szary, A., Kępa, A., Żarowski, M., Hojan, K., Łukowicz, M., et al. (2024) · International Journal of Molecular Sciences · doi:10.3390/ijms25115749 · Cited by 35
“However limitations, narrative reviews are valuable in providing a holistic understanding These limitations include potential biases, subjectivity in data interpretation, lack of transparency in the review process, and the absence of a systematic approach to evidence synthesis [84].”
BackgroundMusa, M. K., Akdur, G., Hanratty, B., Kelly, S. A., Gordon, A. L., Peryer, G., Spilsbury, K., Killett, A., Burton, J. K., Meyer, J. E., et al. (2020) · BMJ Open · doi:10.1136/bmjopen-2020-040397 · Cited by 24
Development of a program theory for shared decision-making: a realist review protocol
Groot, G., Waldron, T., Carr, T., McMullen, L. M., Bandura, L.-A., Neufeld, S.-M., & Duncan, V. (2017) · Systematic Reviews · doi:10.1186/s13643-017-0508-5 · Cited by 28
“odology [37, 41], (4) extract data, (5) identify relevant”
BackgroundGrünwald, L. M., Duddy, C., Byng, R., Crellin, N. E., & Moncrieff, J. (2021) · BMC Psychiatry · doi:10.1186/s12888-021-03355-3 · Cited by 22
“The review has been conducted and reported following the RAMESES standards [18, 19].”
Coles, E. C., Cheyne, H., & Daniel, B. M. (2015) · Systematic Reviews · doi:10.1186/s13643-015-0068-5 · Cited by 27
“ESES group [42] and will follow the format set out by the RAMESES standards.”
MethodKayes, N. M., Martin, R., Bright, F. A. S., Kersten, P., & Pollock, A. (2019) · European Journal of Physical and Rehabilitation Medicine · doi:10.23736/s1973-9087.19.05793-9 · Cited by 18
How and why do financial incentives contribute to helping people stop smoking? A realist review
Siersbaek, R., Kavanagh, P. M., Ford, J. A., Burke, S., & Parker, S. (2024) · BMC Public Health · doi:10.1186/s12889-024-17967-3 · Cited by 21
“We also were guided by the RAMESES publishing standards in the review and writing process [28].”
Widening access to medicine: A realist review
Bartle, E. K., Carr, S., Olson, R. E., Mogensen, L. L., Hyde, S., Shepherd, N., Hu, W. C. Y., Peña, A. S., Roberts, P., Downes, N., et al. (2025) · Medical Education · doi:10.1111/medu.70017 · Cited by 20
“32 The review was conducted by a 12-person review team from January 2023 to June 2024.”
Lidocaine for postoperative pain after cardiac surgery: a systematic review
Boswell, M. R., Moman, R. N., Burtoft, M. A., Gerdes, H., Martinez, J. S., Gerberi, D., Wittwer, E. D., Murad, M. H., & Hooten, W. M. (2021) · Journal of Cardiothoracic Surgery · doi:10.1186/s13019-021-01549-0 · Cited by 20
Chambers, N., Harvey, G., & Mannion, R. (2017) · Cogent Business & Management · doi:10.1080/23311975.2017.1357348 · Cited by 16
“In our case, using evolving standards in realist review (Wong et al., 2014), the synthesis addresses questions about how boards operate, in what circumstances, and why, and the influence that boards may have on organisational performance.”
MethodBabovič, M., Fu, R.-H., & Monrouxe, L. V. (2019) · BMJ Open · doi:10.1136/bmjopen-2019-029173 · Cited by 17
“…the literature deeper for theoretical elements which might help to explain new findings, or re-examine certain aspects of the developing theory, 32 we expect to undertake additional inquires such as: (1) hand searching relevant journals (related to e-learning, e-portfolio or feedback in an…”
BackgroundMethodInfluentialStolee, P., Elliott, J. B., McNeil, H., Boscart, V., Heckman, G. A., Hutchinson, R., Hedley, M. R., Glouberman, S., & Judd, M. (2015) · BMJ Open · doi:10.1136/bmjopen-2015-008190 · Cited by 16
“14 16 Bespoke abstraction forms will be developed based on the scope clari fi cation phase and relevant theories of engagement.”
BackgroundRemediating doctors’ performance to restore patient safety: a realist review protocol
Price, T., Brennan, N., Cleland, J., Prescott‐Clements, L., Wanner, A., Withers, L., Wong, G., & Archer, J. C. (2018) · BMJ Open · doi:10.1136/bmjopen-2018-025943 · Cited by 16
“The study findings will be written up according to the Realist And Meta-narrative Evidence Syntheses—Evolving Standards (RAMESES) quality and publication standards.(22) This protocol is reported according to Preferred Reporting Items for Systematic Reviews and Meta-Analyses Protocols guidelines.”
MethodSelf-citationHow does integrated knowledge translation work? A realist review
Kothari, A., Sibbald, S. L., McCutcheon, C., Berta, W., Urquhart, R., Horsley, T., Comer, L., & Graham, I. D. (2025) · Health Research Policy and Systems · doi:10.1186/s12961-025-01374-0 · Cited by 17
Calderón‐Larrañaga, S., Clinch, M., Greenhalgh, T., & Finer, S. (2021) · BMJ Open · doi:10.1136/bmjopen-2020-042303 · Cited by 15
Self-citationThe usage of data in NHS primary care commissioning: a realist review
Jager, A., Wong, G., Papoutsi, C., & Roberts, N. W. (2023) · BMC Medicine · doi:10.1186/s12916-023-02949-w · Cited by 22
Self-citationDemand management for planned care: a realist synthesis
Pawson, R., Greenhalgh, J., & Brennan, C. A. (2016) · Health Services and Delivery Research · doi:10.3310/hsdr04020 · Cited by 22
“A comprehensive overview on methodological guidance, publication standards, training materials and a bibliography of earlier realist reviews is available as part of a previous Health Services and Delivery Research (HSDR) report.(4,5) Readers are referred to these sources for a detailed account of the method.”
BackgroundMethodSelf-citationPromises and Perils of Group Clinics for Young People Living With Diabetes: A Realist Review
Papoutsi, C., Colligan, G., Hagell, A., Hargreaves, D. S., Marshall, M., Vijayaraghavan, S., Greenhalgh, T., & Finer, S. (2019) · Diabetes Care · doi:10.2337/dc18-2005 · Cited by 17
“available in the RAMESES training materials (15).”
MethodSelf-citationMeiling, J. B., Barndt, B. S., Ha, C. T., Eubanks, J. E., Schappell, J. B., Raum, G. M., Khan, S. A., Prokop, L. J., Conger, A., McCormick, Z. L., et al. (2022) · Interventional Pain Medicine · doi:10.1016/j.inpm.2022.100072 · Cited by 17
Power, J., Dada, S., Booth, A., Brún, A. D., & Gilmore, B. (2024) · Cochrane Evidence Synthesis and Methods · doi:10.1002/cesm.12073 · Cited by 18
Ouellet, S., Gallani, M. C. B. J., Gélinas, C., Fontaine, G., Archambault, P. M., Mercier, É., Severino, F., & Berube, M. (2022) · Nursing Open · doi:10.1002/nop2.1550 · Cited by 21
Duprez, V., Dhont, L., van der Cingel, M., Hafsteinsdøttir, T. B., & Malfait, S. (2023) · Journal of Advanced Nursing · doi:10.1111/jan.15902 · Cited by 23
Blane, D., Macdonald, S., Morrison, D. S., & O’Donnell, C. A. (2015) · Systematic Reviews · doi:10.1186/s13643-015-0046-y · Cited by 18
“It is expected that a realist synthesis will produce a description of positive and negative context-mechanismoutcome (CMO) configurations, explore patterns among these CMO configurations, and develop and test one or more middle-range theories that explain how and why these configurations relate to each other, drawing…”
BackgroundMirzoev, T. N., Kane, S. S., Azdi, Z. A., Ebenso, B., Chowdhury, A. A., & Huque, R. (2021) · BMJ Global Health · doi:10.1136/bmjgh-2020-004357 · Cited by 17
An Evidence-Based Theory About PRO Use in Kidney Care: A Realist Synthesis
Schick‐Makaroff, K., Levay, A., Thompson, S., Flynn, R., Sawatzky, R. G., Thummapol, O., Klarenbach, S. W., Karimi-Dehkordi, M., & Greenhalgh, J. (2021) · Patient · doi:10.1007/s40271-021-00530-2 · Cited by 15
Treacy, M., Wong, G., Odell, M., & Roberts, N. W. (2022) · BMJ Open · doi:10.1136/bmjopen-2022-062154 · Cited by 18
Self-citationWarner, G., Baird, L. G., McCormack, B. G., Urquhart, R., Lawson, B. J., Tschupruk, C., Christian, E., Weeks, L. E., Kumanan, K., & Sampalli, T. (2021) · BMC Palliative Care · doi:10.1186/s12904-021-00781-8 · Cited by 18
“Finally, the Quality Standards for Realist Synthesis form [31] was completed and discussed within”
BackgroundMethodOwen-Boukra, E., Cai, Z., Duddy, C., Fudge, N., Hamer-Hunt, J., Husson, F., Mahtani, K. R., Ogden, M., Swinglehurst, D., Turner, M., et al. (2024) · Journal of Health Services Research & Policy · doi:10.1177/13558196241290923 · Cited by 14
“16 The review was carried out in accordance with the Realist and Meta-narrative Evidence Synthesis: Evolving Standards (RAMESES) quality and reporting standards 17,18 and has been registered on PROSPERO (CRD42022314280).”
Self-citationPapoutsi, C., Hargreaves, D. S., Colligan, G., Hagell, A., Patel, A. K., Campbell-Richards, D., Viner, R., Vijayaraghavan, S., Marshall, M., Greenhalgh, T., et al. (2017) · BMJ Open · doi:10.1136/bmjopen-2017-017363 · Cited by 14
Self-citationFraming the time while designing and conducting reviews: A Focused Mapping Review and Synthesis
Palese, A., Mansutti, I., Visintini, E., Caruzzo, D., Moreale, R., Longhini, J., & Danielis, M. (2021) · Journal of Clinical Nursing · doi:10.1111/jocn.16180 · Cited by 13
Research protocol: a realist synthesis of contestability in community-based mental health markets
Durham, J., & Bains, A. (2015) · Systematic Reviews · doi:10.1186/s13643-015-0025-3 · Cited by 15
“For example, compared to systematic reviews, realist reviews are harder to reproduce as selecting papers and identifying candidate theories requires judgment that is often based on a mixture of experience, intuition and prudence to identify those with greatest relevance [23,30].”
BackgroundInfluentialChan, T. M., Thoma, B., Hall, A. K., Murnaghan, A., Ting, D. K., Hagel, C., Weersink, K., Camorlinga, P., McEwen, J., Bhanji, F., et al. (2017) · Canadian Journal of Emergency Medicine · doi:10.1017/cem.2017.30 · Cited by 10
Bunn, F., Goodman, C. M., Russell, B., Wilson, P. M., Manthorpe, J., Rait, G., Hodkinson, I., & Durand, M. (2018) · Health Services and Delivery Research · doi:10.3310/hsdr06280 · Cited by 16
Metadata
- Authors
- Wong, G., Greenhalgh, T., Westhorp, G., & Pawson, R.
- Affiliations
- University of London (GB)
- Centre for Primary Care and Public Health, Queen Mary University of London, London, UK
- Community Matters, Mount Torrens, SA, Australia
- University of Leeds (GB)
- Corresponding author
- Wong, Geoff
- Venue
- Health Services and Delivery Research
- Published
- 2014-09
- Volume
- 2
- Issue
- 30
- Pages
- 1-252
- ISSN
- 2050-4349 (ISSN-L)
- 2050-4357
- OpenAlex
- W2072084577
- Cited by
- 276
- References
- 97
- Citations by year
- 2026: 26 · 2025: 40 · 2024: 31 · 2023: 24 · 2022: 36 · 2021: 40 · 2020: 15 · 2019: 20 · 2018: 14 · 2017: 16 · 2016: 8 · 2015: 6
- FWCI
- 2.75
- Type
- journal article
- Language
- en
- Open access
- diamond
- doi.org
- Locations
- Health Services and Delivery Research (publishedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- National Institute for Health Research (10/1008/07)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Public Administration › Evaluation and Performance Assessment
- Health Sciences › Health Professions › General Health Professions › Health Policy Implementation Science
- Keywords
- realist review, meta-narrative review, methodological recommendations, publication standards, training materials, expert consensus, Delphi method, review quality, research training, policy-relevant research, research capacity building
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.