The DOI matches its Crossref record; 38 of the 40 references it cites were found in a registry or an index.
A realistic evaluation: the case of protocol-based care
Open accessAbstract
BACKGROUND: 'Protocol based care' was envisioned by policy makers as a mechanism for delivering on the service improvement agenda in England. Realistic evaluation is an increasingly popular approach, but few published examples exist, particularly in implementation research. To fill this gap, within this paper we describe the application of a realistic evaluation approach to the study of protocol-based care, whilst sharing findings of relevance about standardising care through the use of protocols, guidelines, and pathways. METHODS: Situated between positivism and relativism, realistic evaluation is concerned with the identification of underlying causal mechanisms, how they work, and under what conditions. Fundamentally it focuses attention on finding out what works, for whom, how, and in what circumstances. RESULTS: In this research, we were interested in understanding the relationships between the type and nature of particular approaches to protocol-based care (mechanisms), within different clinical settings (context), and what impacts this resulted in (outcomes). An evidence review using the principles of realist synthesis resulted in a number of propositions, i.e., context, mechanism, and outcome threads (CMOs). These propositions were then 'tested' through multiple case studies, using multiple methods including non-participant observation, interviews, and document analysis through an iterative analysis process. The initial propositions (conjectured CMOs) only partially corresponded to the findings that emerged during analysis. From the iterative analysis process of scrutinising mechanisms, context, and outcomes we were able to draw out some theoretically generalisable features about what works, for whom, how, and what circumstances in relation to the use of standardised care approaches (refined CMOs). CONCLUSIONS: As one of the first studies to apply realistic evaluation in implementation research, it was a good fit, particularly given the growing emphasis on understanding how context influences evidence-based practice. The strengths and limitations of the approach are considered, including how to operationalise it and some of the challenges. This approach provided a useful interpretive framework with which to make sense of the multiple factors that were simultaneously at play and being observed through various data sources, and for developing explanatory theory about using standardised care approaches in practice.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Figures
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Rather than identifying simple cause-and-effect relationships, RE is concerned with finding out about what mechanisms work, in what conditions, why, to produce which outcomes and for whom.124”
From ReseArch with Patient and Public invOlvement: a RealisT evaluation – the RAPPORT study (2015) · cites this paper as [124]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
(1984) · Cited by 81,983
Hartley, J. (2004) · Essential Guide to Qualitative Methods in Organizational Research · doi:10.4135/9781446280119.n26 · Cited by 15,461
(1993) · Cited by 7,856
Diffusion of Innovations in Service Organizations: Systematic Review and Recommendations
GREENHALGH, T., ROBERT, G., MACFARLANE, F., BATE, P., & KYRIAKIDOU, O. (2004) · The Milbank Quarterly · doi:10.1111/j.0887-378x.2004.00325.x · Cited by 7,448
Realist review - a new method of systematic review designed for complex policy interventions
Pawson, R., Greenhalgh, T., Harvey, G., & Walshe, K. (2005) · Journal of Health Services Research & Policy · doi:10.1258/1355819054308530 · Cited by 3,190
Pawson, R. (2006) · doi:10.4135/9781849209120 · Cited by 1,791
The new NHS: modern dependable
Warwick, C. (1998) · British Journal of Midwifery · doi:10.12968/bjom.1998.6.4.265 · Cited by 1,228
Getting evidence into practice: the meaning of `context'
McCormack, B., Kitson, A., Harvey, G., Rycroft‐Malone, J., Titchen, A., & Seers, K. (2002) · Journal of Advanced Nursing · doi:10.1046/j.1365-2648.2002.02150.x · Cited by 765
Ingredients for change: revisiting a conceptual framework
Rycroft-Malone, J. (2002) · Quality and Safety in Health Care · doi:10.1136/qhc.11.2.174 · Cited by 701
An exploration of the factors that influence the implementation of evidence into practice
Rycroft‐Malone, J., Harvey, G., Seers, K., Kitson, A., McCormack, B., & Titchen, A. (2004) · Journal of Clinical Nursing · doi:10.1111/j.1365-2702.2004.01007.x · Cited by 623
Research works printed but not linked: 1
- Pawson R, Tilley N: Realistic evaluation. 1997, London: Sage
Cited by
Sorted by citations
Items 101–173 of 173
Fletcher-Hildebrand, S., Alimezelli, H. T., Carr, T., Lawson, K. L., Ali, A., & Groot, G. (2021) · BMJ Open · doi:10.1136/bmjopen-2020-044522 · Cited by 8
Sachidanandan, G., Bechard, L., Hodgson, K. J., & Sud, A. (2022) · International Journal of Drug Policy · doi:10.1016/j.drugpo.2022.103807 · Cited by 9
Magnin, M., Jeannot, E., Voahangy, R., & Stoll, B. (2018) · Acta Paediatrica · doi:10.1111/apa.14267 · Cited by 5
A bottom‐up framework for nurses' protocol‐based care decision‐making
Vázquez‐Calatayud, M., Pumar‐Méndez, M. J., & Oroviogoicoechea, C. (2024) · Nursing Open · doi:10.1002/nop2.2232 · Cited by 5
Hamadeh, S., Willetts, G., & Garvey, L. (2024) · Journal of Advanced Nursing · doi:10.1111/jan.16500 · Cited by 7
Sustainability of an outcome measure in outpatient stroke rehabilitation: A realist evaluation
Ataman, R., Ahmed, S., Berta, W. B., & Thomas, A. (2024) · Journal of Evaluation in Clinical Practice · doi:10.1111/jep.13972 · Cited by 5
Mutua, M. N., & Nakidde, C. (2024) · International Journal of Qualitative Methods · doi:10.1177/16094069241301984 · Cited by 5
Tulloch, A. D., Soper, B., Görzig, A., Pettit, S., Koeser, L., Polling, C., Watson, A., Khondoker, M. R., Rose, D. S., McCrone, P., et al. (2016) · Health Services and Delivery Research · doi:10.3310/hsdr04090 · Cited by 5
Harding, A. J. E., Hean, S., Parker, J., & Hemingway, A. (2020) · Social Policy and Society · doi:10.1017/s1474746419000472 · Cited by 5
Chorwe‐Sungani, G., Mwagomba, M., Chirwa, E. M., Jere, D. L., & Chipps, J. A. (2022) · BMC Psychiatry · doi:10.1186/s12888-022-04195-5 · Cited by 6
Understanding assessment on a hospital ward for older people AS PER PDF: A qualitative study
Wiltjer, H., Seers, K., & Tutton, E. (2019) · Journal of Advanced Nursing · doi:10.1111/jan.13930 · Cited by 4
The PHARMS (Patient Held Active Record of Medication Status) feasibility study: a research proposal
Walsh, E. K., Sahm, L. J., Kearney, P. M., Smithson, H., Kerins, D. M., Ngwa, C., Fitzgerald, C., Carthy, S. M., Connolly, E., Dalton, K., et al. (2018) · BMC Research Notes · doi:10.1186/s13104-017-3118-3 · Cited by 4
Schreurs, R. H. P., Joore, M. A., Bruijn‐Geraets, D. P. D., Cate, H. T., & ten Cate‐Hoek, A. J. (2022) · PLoS ONE · doi:10.1371/journal.pone.0272566 · Cited by 4
14. Théorie et pratique de l’approche Realist pour l’évaluation des programmes
Ridde, V., Robert, É., Guichard, A., Blaise, P., & van Olmen, J. (2012) · Les Presses de l'Université de Montréal eBooks · doi:10.4000/books.pum.5999 · Cited by 4
Screening and alcohol brief interventions in antenatal care: a realistic evaluation
Doi, L. K. (2012) · Stirling Online Research Repository (University of Stirling) · Cited by 4
Collier, A., Appachoo, S., Frey, R. A., Jones, V., Birtwistle, J. C., Allsop, M. J., & Bloomfield, K. (2025) · BMC Palliative Care · doi:10.1186/s12904-025-01932-x · Cited by 3
Andersson, U., Söderholm, H. M., Andersson, H., Sundström, B. W., Hagiwara, M. A., & Puača, G. (2023) · Paramedicine · doi:10.1177/27536386231189011 · Cited by 3
Factors influencing change in clinical practice
Rajasekhar, P. T., Rees, C. J., Nixon, C., East, J. E., & Brown, S. R. (2016) · International Journal of Health Care Quality Assurance · doi:10.1108/ijhcqa-09-2014-0093 · Cited by 3
Modified realist evaluation of a complex, multi-centred, multi-intervention programme
Greenhalgh, C., Williams, G., Harrison, A., Garrow, A. P., Mitchell, S., & Verma, A. (2023) · Journal of Public Health · doi:10.1093/pubmed/fdad029 · Cited by 3
Chorwe‐Sungani, G., Mwagomba, M., Kulisewa, K., Chirwa, E. M., Jere, D. L., & Chipps, J. A. (2021) · Pilot and Feasibility Studies · doi:10.1186/s40814-021-00775-6 · Cited by 4
Brady, C., Burgess, R. A., & Osrin, D. (2022) · SSM - Mental Health · doi:10.1016/j.ssmmh.2022.100066 · Cited by 3
Kitson, A. A., Wiechula, R., Zeitz, C., Marcoionni, D., Page, T., & Silverston, H. (2011) · Adelaide Research & Scholarship (AR&S) (University of Adelaide) · Cited by 3
Beutel, D., Tangen, D., & Carrington, S. B. (2022) · Evaluation and Program Planning · doi:10.1016/j.evalprogplan.2022.102201 · Cited by 2
Towards defining and advancing storytelling approach in an indigenous realist evaluation
Mutua, M. N. (2025) · Evaluation Journal of Australasia · doi:10.1177/1035719x251336866 · Cited by 3
Developing and implementing a protocol for major haemorrhage
Hargreaves, J. (2014) · British Journal of Nursing · doi:10.12968/bjon.2014.23.21.1140 · Cited by 2
Masso, M. R. (2011) · Research Online (University of Wollongong) · Cited by 2
Avvenning fra respirator – Protokoll og evidens
Værland, I. E., & Kristoffersen, K. (2011) · Nordisk Sygeplejeforskning · doi:10.18261/issn1892-2686-2011-04-04 · Cited by 2
Windle, K. (2015) · Lincoln Repository (University of Lincoln) · Cited by 2
Lidegaard, M., Olsen, K. B., & Legg, S. J. (2019) · Evaluation and Program Planning · doi:10.1016/j.evalprogplan.2019.01.002 · Cited by 2
Realism and rhetoric in the evaluation of a new care model
Grimwood, T. (2019) · Journal of Integrated Care · doi:10.1108/jica-11-2018-0067 · Cited by 2
Safi‐Keykaleh, M., Khorasani‐Zavareh, D., Ghomian, Z., Nateghinia, S., Safarpour, H., & Mohammadi, R. (2021) · Journal of Education and Health Promotion · doi:10.4103/jehp.jehp_1566_20 · Cited by 2
Plessis, A. H. D., van Rooyen, D. R. M., & ten Ham‐Baloyi, W. (2021) · Health SA Gesondheid · doi:10.4102/hsag.v26i0.1631 · Cited by 2
Development of competency standards to inform intensive care nursing practice
Schmollgruber, S. (2015) · Cited by 2
System‐failing creativity in health care
Horck, S., Gifford, R. E., Fleuren, B. P. I., Rathert, C., Porter, T. H., Rauf, A., & Lee, Y. S. H. (2024) · Learning Health Systems · doi:10.1002/lrh2.10437 · Cited by 1
Triwahyudi, D., Hafidz, F., & Trisnantoro, L. (2021) · Jurnal Manajemen Pelayanan Kesehatan (The Indonesian Journal of Health Service Management) · doi:10.22146/jmpk.v24i02.4117 · Cited by 1
Bunch, J. L. (2014) · doi:10.17077/etd.f0vo2dmt · Cited by 1
Exploring the assessment process on a ward for older people : a constructivist grounded theory
Wiltjer, H. (2026) · Warwick Research Archive Portal (University of Warwick) · doi:10.82444/warw.33754582 · Cited by 1
Third sector telephone housing options service for older people:a realist evaluation
Harding, A. J. E. (2018) · Bournemouth University Research Online (Bournemouth University) · Cited by 1
Barker, R. (2020) · Kent Academic Repository (University of Kent) · Cited by 1
Craig, A.-M. (2020) · St Andrews Research Repository (St Andrews Research Repository) · doi:10.17630/sta/16 · Cited by 2
A Realist Evaluation of a Recovery-Oriented Inpatient Service
Pritchard, A. (2021) · Cronfa (Swansea University) · doi:10.23889/suthesis.57769 · Cited by 1
Owen-Boukra, E. (2021) · Cronfa (Swansea University) · doi:10.23889/suthesis.58701 · Cited by 1
Evaluating criminal justice interventions in the field of domestic violence: a realist approach
Taylor, H. A. (2014) · Cited by 1
Yano, E. M. (2014) · Journal of General Internal Medicine · doi:10.1007/s11606-014-2820-2
Drury, V., Chiang, P. P., Tey, C. S., Soon, H. J., & Aw, A. T. (2014) · International Journal of Ophthalmic Practice · doi:10.12968/ijop.2014.5.5.182
Sinclair, L. (2013) · OPUS - Open Publications of UTS Scholars (University of Technology Sydney)
The Sustainability of Innovations in Hospitals: A Look at Rapid Response Teams
Stolldorf, D. P. (2019) · Carolina Digital Repository (University of North Carolina at Chapel Hill) · doi:10.17615/nec5-v128
Users who downloaded this article also downloaded
Martinez, F., Lenssen, G., Nijhof, A. H. J., Roger, L., Kievit, H., Lenssen, J., Dentchev, N. A., Bakker, A. J., & Schaveling, J. (2014)
An implementation post-mortem : a study of coordinated care at the end of life
Holdsworth, L. M. (2016) · Kent Academic Repository (University of Kent)
A realist evaluation of patient involvement in a safer surgery initiative
Roche, D. (2016) · ORCA Online Research @Cardiff (Cardiff University)
Sykes, P. (2015) · UTAS Research Repository · doi:10.25959/23238803
Realistically Evaluating Small Scale SFD Programmes Professionalisation and Praxis
Harris, K. (2017) · Southampton Solent University Digital Library · doi:10.13140/rg.2.2.12856.60161
de Grace (2015) · ERA: Education and Research Archive (University of Alberta) · doi:10.7939/r3wm1451w
(2013) · Maternal and Child Nutrition · doi:10.1111/mcn.12093
Innovations in managing at-risk and deteriorating adult patients at home
Scott, S. (2019) · Primary Health Care · doi:10.7748/phc.2019.e1463
Iyengar, S. (2019) · The University of Queensland · doi:10.14264/uql.2019.153
Strangers in the house : the impact of domiciliary care for older people
Raithby, M. (2016)
Hoe weet je wat werkt? Over de zoektocht naar bruikbare kennis voor beleidseffectiviteit
van Eersel, G. G. (2021) · Beleidsonderzoek Online · doi:10.5553/bo/221335502021007
Towards improved organisation of care
Schreurs, R. H. P. (2023) · doi:10.26481/dis.20230118rs
Downey, J., Fornasiero, M., Cooper, S., Bassett, L., Doherty, M., Fong, A. D., Bradley, N., & Cornwall, J. (2023) · Palliative Medicine · doi:10.1177/02692163231152524
Realistisk evaluering – en metodologi
Søndergaard, S. F., Beedholm, K., Kolbæk, R., Andersen, A. B., & Frederiksen, K. (2023) · Klinisk Sygepleje · doi:10.18261/ks.37.1.4
Hudon, C., Chouinard, M., Doucet, S. A., Piccinini‐Vallis, H., Fairman, K., Sampalli, T., Zed, J., Brodeur, M., Chênevert, D., Dépelteau, A., et al. (2024) · BMJ Open · doi:10.1136/bmjopen-2023-083783
Mouchaers, I. (2024) · doi:10.26481/dis.20240913im
Bardach, D. R. (2015) · UKnowledge (University of Kentucky)
Chorwe‐Sungani, G., Mwagomba, M., Kulisewa, K., Chirwa, E., Jere, D. L., & Chipps, J. (2021) · Figshare · doi:10.6084/m9.figshare.13642420
Chorwe‐Sungani, G., Mwagomba, M., Kulisewa, K., Chirwa, E., Jere, D. L., & Chipps, J. (2021) · Figshare · doi:10.6084/m9.figshare.13642414
Chorwe‐Sungani, G., Mwagomba, M., Kulisewa, K., Chirwa, E., Jere, D. L., & Chipps, J. (2021) · Figshare · doi:10.6084/m9.figshare.13642423
Chorwe‐Sungani, G., Mwagomba, M., Kulisewa, K., Chirwa, E., Jere, D. L., & Chipps, J. (2021) · Figshare · doi:10.6084/m9.figshare.13642426
Chorwe‐Sungani, G., Mwagomba, M., Kulisewa, K., Chirwa, E., Jere, D. L., & Chipps, J. (2021) · Figshare · doi:10.6084/m9.figshare.13642417
Chorwe‐Sungani, G., Mwagomba, M., Kulisewa, K., Chirwa, E., Jere, D. L., & Chipps, J. (2021) · Figshare · doi:10.6084/m9.figshare.13642411
Metadata
- Authors
- Rycroft-Malone, J., Fontenla, M., Bick, D., & Seers, K.
- Affiliations
- University of Warwick (GB)
- King's College London (GB)
- Venue
- Implementation Science
- Published
- 2010-05-26
- Volume
- 5
- Issue
- 1
- Pages
- 38-38
- ISSN
- 1748-5908 (ISSN-L)
- PMC
- PMC2889857
- OpenAlex
- W2032357725
- Cited by
- 173
- References
- 40
- Citations by year
- 2026: 1 · 2025: 2 · 2024: 8 · 2023: 6 · 2022: 9 · 2021: 18 · 2020: 11 · 2019: 15 · 2018: 17 · 2017: 13 · 2016: 10 · 2015: 18 · 2014: 17 · 2013: 8 · 2012: 10
- FWCI
- 15.95
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Implementation Science (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Warwick Research Archive Portal (University of Warwick) (acceptedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
- Lancaster EPrints (Lancaster University) (submittedVersion)
- Research Portal (King's College London) (submittedVersion)
- Funders
- National Institute for Health and Care Research
- Service Delivery and Organisation Programme
- Topics
- Health Sciences › Health Professions › General Health Professions › Health Policy Implementation Science
- Social Sciences › Public Administration › Evaluation and Performance Assessment
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Clinical practice guidelines implementation
- Keywords
- protocol-based care, realist evaluation, realist synthesis, standardization of care, clinical guidelines, care pathways, implementation research, context-mechanism-outcome, evidence-based practice, health service improvement, causal mechanisms
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review


