The DOI matches its Crossref record; 33 of the 34 references it cites were found in a registry or an index.
RAMESES publication standards: realist syntheses
Abstract
Background: There is growing interest in realist synthesis as an alternative systematic review method. This approach offers the potential to expand the knowledge base in policy‐relevant areas – for example, by explaining the success, failure or mixed fortunes of complex interventions. No previous publication standards exist for reporting realist syntheses. This standard was developed as part of the RAMESES (Realist And ME ta‐narrative Evidence Syntheses: Evolving Standards) project. The project's aim is to produce preliminary publication standards for realist systematic reviews. Design: A mixed method study synthesising data between 2011–2012 from a literature review, online Delphi panel and feedback from training, workshops and email list. Methods: We: (a) collated and summarized existing literature on the principles of good practice in realist syntheses; (b) considered the extent to which these principles had been followed by published syntheses, thereby identifying how rigour may be lost and how existing methods could be improved; (c) used a three‐round online Delphi method with an interdisciplinary panel of national and international experts in evidence synthesis, realist research, policy and/or publishing to produce and iteratively refine a draft set of methodological steps and publication standards; (d) provided real‐time support to ongoing realist syntheses and the open‐access RAMESES online discussion list to capture problems and questions as they arose; and (e) synthesized expert input, evidence syntheses and real‐time problem analysis into a definitive set of standards. Results: We identified 35 published realist syntheses, provided real‐time support to 9 ongoing syntheses and captured questions raised in the RAMESES discussion list. Through analysis and discussion within the project team, we summarized the published literature and common questions and challenges into briefing materials for the Delphi panel, comprising 37 members. Within 3 rounds this panel had reached consensus on 19 key publication standards, with an overall response rate of 91%. Conclusions: This project used multiple sources to develop and draw together evidence and expertise in realist synthesis. For each item we have included an explanation for why it is important and guidance on how it might be reported. Realist synthesis is a relatively new method for evidence synthesis and as experience and methodological developments occur, we anticipate that these standards will evolve to reflect further methodological developments. We hope that these standards will act as a resource that will contribute to improving the reporting of realist syntheses.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Wong et al33 states that realist reviews reveal ‘what policy makers or practitioners might put in place to change the context or provide resources in such a way as to most likely trigger the right mechanism(s) to produce the desired outcome’.”
From mHealth text and voice communication for monitoring people with chronic diseases in low-resource settings: a realist review (2018) · cites this paper as [33]
Citations
Cites
Liberati, A., Altman, D. G., Tetzlaff, J., Mulrow, C., Gøtzsche, P. C., Ioannidis, J. P. A., Clarke, M., Devereaux, P. J., Kleijnen, J., & Moher, D. (2009) · PLoS Medicine · doi:10.1371/journal.pmed.1000100 · Cited by 28,164
Liberati, A., Altman, D. G., Tetzlaff, J., Mulrow, C., Gotzsche, P. C., Ioannidis, J. P. A., Clarke, M., Devereaux, P. J., Kleijnen, J., & Moher, D. (2009) · BMJ · doi:10.1136/bmj.b2700 · Cited by 17,648
Moher, D., Hopewell, S., Schulz, K. F., Montori, V., Gotzsche, P. C., Devereaux, P. J., Elbourne, D., Egger, M., & Altman, D. G. (2010) · BMJ · doi:10.1136/bmj.c869 · Cited by 8,883
AGREE II: advancing guideline development, reporting and evaluation in health care
Brouwers, M. C., Kho, M. E., Browman, G. P., Burgers, J. S., Cluzeau, F., Feder, G., Fervers, B., Graham, I. D., Grimshaw, J., Hanna, S. E., et al. (2010) · Canadian Medical Association Journal · doi:10.1503/cmaj.090449 · Cited by 3,790
Realist review - a new method of systematic review designed for complex policy interventions
Pawson, R., Greenhalgh, T., Harvey, G., & Walshe, K. (2005) · Journal of Health Services Research & Policy · doi:10.1258/1355819054308530 · Cited by 3,190
Self-citationMethods for the synthesis of qualitative research: a critical review
Barnett-Page, E., & Thomas, J. (2009) · BMC Medical Research Methodology · doi:10.1186/1471-2288-9-59 · Cited by 2,386
Synthesising qualitative and quantitative evidence: A review of possible methods
Dixon-Woods, M., Agarwal, S., Jones, D., Young, B., & Sutton, A. (2005) · Journal of Health Services Research & Policy · doi:10.1177/135581960501000110 · Cited by 2,132
Pawson, R. (2006) · doi:10.4135/9781849209120 · Cited by 1,791
Self-citationJAGOSH, J., MACAULAY, A. C., PLUYE, P., SALSBERG, J., BUSH, P. L., HENDERSON, J., SIRETT, E., WONG, G., CARGO, M., HERBERT, C. P., et al. (2012) · The Milbank Quarterly · doi:10.1111/j.1468-0009.2012.00665.x · Cited by 1,268
Self-citationAn Introduction to Systematic Reviews
Gough, D., Oliver, S., & Thomas, J. (2017) · doi:10.4135/9781036234942 · Cited by 1,239
Unpacking black boxes: mechanisms and theory building in evaluation
Astbury, B., & Leeuw, F. L. (2010) · American Journal of Evaluation · doi:10.1177/1098214010371972 · Cited by 850
Realist synthesis: illustrating the method for implementation research
Rycroft-Malone, J., McCormack, B., Hutchinson, A. M., DeCorby, K., Bucknall, T. K., Kent, B., Schultz, A., Snelgrove-Clarke, E., Stetler, C. B., Titler, M., et al. (2012) · Implementation Science · doi:10.1186/1748-5908-7-33 · Cited by 742
Evidence-based policy: the promise of ‘realist synthesis’
Pawson, R. (2002) · Evaluation · doi:10.1177/135638902401462448 · Cited by 729
Self-citationRealist synthesis - an introduction
Pawson, R., Greenhalgh, T., Harvey, G., & Walshe, K. (2004) · UCL Discovery (University College London) · Cited by 570
Self-citationIntroduction to systematic reviews
O’Connor, E., Whitlock, E. P., & Spring, B. J. (2011) · PsycEXTRA Dataset · doi:10.1037/e657462011-001 · Cited by 464
Internet-based medical education: a realist review of what works, for whom and in what circumstances
Wong, G., Greenhalgh, T., & Pawson, R. (2010) · BMC Medical Education · doi:10.1186/1472-6920-10-12 · Cited by 457
Self-citationCan We Systematically Review Studies That Evaluate Complex Interventions?
Shepperd, S., Lewin, S., Straus, S., Clarke, M., Eccles, M. P., Fitzpatrick, R., Wong, G., & Sheikh, A. (2009) · PLoS Medicine · doi:10.1371/journal.pmed.1000086 · Cited by 408
Self-citationRealist methods in medical education research: what are they and what can they contribute?
Wong, G., Greenhalgh, T., Westhorp, G., & Pawson, R. (2011) · Medical Education · doi:10.1111/j.1365-2923.2011.04045.x · Cited by 405
Self-citationOn Theoretical Sociology: Five Essays, Old and New
Merton, R. Κ. (1967) · Cited by 347
Kastner, M., Tricco, A. C., Soobiah, C., Lillie, E., Perrier, L., Horsley, T., Welch, V., Cogo, E., Antony, J., & Straus, S. E. (2012) · BMC Medical Research Methodology · doi:10.1186/1471-2288-12-114 · Cited by 345 · cited 2×
“At present there is lack of clarity on which methods are best suited for which questions or problems and this has been the subject of debate (Dixon-Woods et al. 2005, Pope et al. 2007, Barnett-Page & Thomas 2009, Gough et al. 2012) and further research (Kastner et al. 2012).”
Protocol - realist and meta-narrative evidence synthesis: Evolving Standards (RAMESES)
Greenhalgh, T., Wong, G., Westhorp, G., & Pawson, R. (2011) · BMC Medical Research Methodology · doi:10.1186/1471-2288-11-115 · Cited by 279
Self-citationDieleman, M. A., Gerretsen, B., & van der Wilt, G. J. (2009) · Health Research Policy and Systems · doi:10.1186/1478-4505-7-7 · Cited by 247
Davidoff, F., Batalden, P. B., Stevens, D., Ogrinc, G. S., Mooney, S. E., & for the SQUIRE development group (2009) · BMJ · doi:10.1136/bmj.a3152 · Cited by 155
Known Knowns, Known Unknowns, Unknown Unknowns
Pawson, R., Wong, G., & Owen, L. (2011) · American Journal of Evaluation · doi:10.1177/1098214011403831 · Cited by 139
Self-citationKane, S. S., Gerretsen, B., Scherpbier, R. W., Poz, M. R. D., & Dieleman, M. A. (2010) · BMC Health Services Research · doi:10.1186/1472-6963-10-286 · Cited by 131
Publication Guidelines for Improvement Studies in Health Care: Evolution of the SQUIRE Project
Davidoff, F., Batalden, P. B., Stevens, D., Ogrinc, G. S., Mooney, S. E., & for the SQUIRE Development Group* (2008) · Annals of Internal Medicine · doi:10.7326/0003-4819-149-9-200811040-00009 · Cited by 130
Synthesizing Qualitative and Quantitative Health Research: A Guide to Methods
Pope, C. J., Mays, N., & Popay, J. (2007) · ePrints Soton (University of Southampton) · Cited by 113
Jackson, L. A., Langille, L. L., Lyons, R., Hughes, J. R., Martin, D. H., & Winstanley, V. (2009) · Health & Place · doi:10.1016/j.healthplace.2009.03.003 · Cited by 88
Policy guidance on threats to legislative interventions in public health: a realist synthesis
Wong, G., Pawson, R., & Owen, L. (2011) · BMC Public Health · doi:10.1186/1471-2458-11-222 · Cited by 67
Self-citationProtocol: a realist review of user fee exemption policies for health services in Africa
Robert, É., Ridde, V., Marchal, B., & Fournier, P. F. (2012) · BMJ Open · doi:10.1136/bmjopen-2011-000706 · Cited by 47
Realist review and synthesis of retention studies for health workers in rural and remote areas
Dieleman, M. A., Kane, S. S., Zwanikken, P. A. C., & Gerretsen, B. (2011) · Data Archiving and Networked Services (DANS) · Cited by 33
Pearson, M., & Coomber, R. (2009) · Addiction · doi:10.1111/j.1360-0443.2009.02713.x · Cited by 18
The Internet in Medical Education: A Worked Example of a Realist Review
Wong, G. (2011) · Wiley eBooks · doi:10.1002/9781119959847.ch5 · Cited by 15
Self-citation
Flagged first, then by citations
1 of the 34 references couldn’t be matched to a known paper.
Research works printed but not linked: 1
- Wong G., 2011, Synthesising Qualitative Research: Choosing the Right Approach
Cited by
Sorted by citations
Time to challenge the spurious hierarchy of systematic over narrative reviews?
Greenhalgh, T., Thorne, S., & Malterud, K. (2018) · European Journal of Clinical Investigation · doi:10.1111/eci.12931 · Cited by 1,267
Self-citationWhat’s in a mechanism? Development of a key concept in realist evaluation
Dalkin, S. M., Greenhalgh, J., Jones, D., Cunningham, B., & Lhussier, M. (2015) · Implementation Science · doi:10.1186/s13012-015-0237-x · Cited by 896
RAMESES II reporting standards for realist evaluations
Wong, G., Westhorp, G., Manzano, A., Greenhalgh, J., Jagosh, J., & Greenhalgh, T. (2016) · BMC Medicine · doi:10.1186/s12916-016-0643-1 · Cited by 880
Self-citationPage, M. J., & Moher, D. (2017) · Systematic Reviews · doi:10.1186/s13643-017-0663-8 · Cited by 769
Hong, Q. N., Pluye, P., Bujold, M., & Wassef, M. (2017) · Systematic Reviews · doi:10.1186/s13643-017-0454-2 · Cited by 754
Metadata
- Authors
- Wong, G., Greenhalgh, T., Westhorp, G., Buckingham, J., & Pawson, R.
- Affiliations
- Queen Mary University of London (GB)
- Centre for Primary Care and Public Health Queen Mary University of London UK
- Community Matters Mount Torrens South Australia Australia
- University of Alberta (CA)
- University of Leeds (GB)
- Venue
- Journal of Advanced Nursing
- Publisher
- Wiley
- Published
- 2013-01-29
- Volume
- 69
- Issue
- 5
- Pages
- 1005-1022
- ISSN
- 0309-2402 (ISSN-L)
- 1365-2648
- OpenAlex
- W2005235635
- Cited by
- 385
- References
- 34
- Citations by year
- 2026: 12 · 2025: 7 · 2024: 26 · 2023: 26 · 2022: 26 · 2021: 34 · 2020: 43 · 2019: 44 · 2018: 33 · 2017: 41 · 2016: 35 · 2015: 34 · 2014: 12 · 2013: 11
- FWCI
- 33.78
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Journal of Advanced Nursing (publishedVersion)
- PubMed (publishedVersion)
- Funders
- National Institute for Health and Care Research (NF-SI-0512-10031, 10/1008/07)
- Health Services Research Programme
- Health Services and Delivery Research Programme
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Health Sciences › Health Professions › General Health Professions › Health Policy Implementation Science
- Social Sciences › Public Administration › Evaluation and Performance Assessment
- Keywords
- realist synthesis, publication standards, reporting guidelines, realist research, complex interventions, evidence synthesis, methodological quality, expert consensus, Delphi panel, metanarrative
- MeSH terms
- Delphi Technique, Publishing
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
