The DOI matches its Crossref record; 25 of the 26 references it cites were found in a registry or an index.
Evidence-based Policy: In Search of a Method
Abstract
Evaluation research is tortured by time constraints. The policy cycle revolves quicker than the research cycle, with the result that `real time' evaluations often have little influence on policy making. As a result, the quest for evidence-based policy has turned increasingly to systematic reviews of the results of previous inquiries in the relevant policy domain. However, this shifting of the temporal frame for evaluation is in itself no guarantee of success. Evidence, whether new or old, never speaks for itself. Accordingly, there is debate about the best strategy of marshalling bygone research results into the policy process. This article joins the imbroglio by examining the logic of the two main strategies of systematic review: `meta-analysis' and `narrative review'. Whilst they are often presented as diametrically opposed perspectives, this article argues that they share common limitations in their understanding of how to provide a template for impending policy decisions. This review provides the background for Part II of the article (to be published in the next issue, Evaluation 8[3]), which considers the merits of a new model for evidence-based policy, namely `realist synthesis'.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Our charting approach was akin to a 'narrative review' (Pawson 2002: 171)”
From Scoping studies: towards a methodological framework (2005) · cites this paper as Pawson 2002: 171
Citations
Cites
On the Social Origins of Dictatorship and Democracy
Moore, B. (1984) · doi:10.1007/978-1-349-86027-2_4 · Cited by 2,709
Social Origins of Dictatorship and Democracy: Lord and Peasant in the Making of the Modern World.
Ness, G. D., & Moore, B. (1967) · American Sociological Review · doi:10.2307/2092029 · Cited by 2,272
Sayer, A. (2000) · doi:10.4135/9781446218730 · Cited by 2,259
Meta-analysis of psychotherapy outcome studies.
Smith, M. L., & Glass, G. V. (1977) · American Psychologist · doi:10.1037/0003-066x.32.9.752 · Cited by 2,154
The Comparative Method: Moving Beyond Qualitative and Quantitative Strategies
Ragin, C. C. (2014) · doi:10.1525/9780520957350 · Cited by 2,130
Efficacy of BCG Vaccine in the Prevention of Tuberculosis
Colditz, G. A. (1994) · JAMA · doi:10.1001/jama.1994.03510330076038 · Cited by 2,021
Sayer, A. (2000) · Cited by 1,753
Salas, M. D. L. N. J., & Kaplan, A. (1967) · Revista española de la opinión pública · doi:10.2307/40180865 · Cited by 1,362
Big Structures, Large Processes, Huge Comparisons
Roth, G., & Tilly, C. (1986) · The American Historical Review · doi:10.2307/1858143 · Cited by 1,343
Popay, J., Rogers, A., & Williams, G. (1998) · Qualitative Health Research · doi:10.1177/104973239800800305 · Cited by 987
Research works printed but not linked: 7
- Hunt, M., 1997, How Science Takes Stock
- McGuire, J.B., 1997, Children and Society, 11, 89
- Moore, B., 1966, Social Origins of Dictatorship and Democracy
- Pawson, R., 1997, Realistic Evaluation
- Ragin, C., 1987, The Comparative Method
- Shadish, W., 1991, Foundations of Program Evaluation
- Towner, E., 1996, Reducing Childhood Accidents - The Effectiveness of Health Promotion Interventions: A Literature Review
Cited by
Sorted by citations
Items 301–400 of 403
What makes a difference? Building a foundation for nationally consistent outcome measures
Housing, A., Services, H. W., & Citymission, M. (2013) · Cited by 1
Lemire, S. T., & Christie, C. A. (2018) · Evaluation · doi:10.1177/1356389018763244 · Cited by 1
“…meta-analytic focus on what might be termed “descriptive causation” in many systematic reviews; that is, a focus on determining what works, at the expense of limited attention to explanatory theories about how and why the what works (Miller and Pollock, 1994; Pawson, 2002, 2006; Shadish, 1996).”
BackgroundMethodReviewing Literature for and as Research
D’Souza, N., & Wong, G. (2019) · Innovation and change in professional education · doi:10.1007/978-981-13-3128-2_33 · Cited by 1
Andalecio, A. B. P. (2021) · JOURNAL OF TOURISM CULINARY AND ENTREPRENEURSHIP (JTCE) · doi:10.37715/jtce.v1i2.2201 · Cited by 1
McAneney, H., Shier, H., Gibbs, L. F., Davies, C., Brún, A. D., Tisdall, E. K. M., Corrigan, C., Kelly, A., Owens, J., Okoli, O., et al. (2021) · HRB Open Research · doi:10.12688/hrbopenres.13290.2 · Cited by 2
“, 2018), which enhances the relevance and practical application of the results (Pawson, 2002).”
InfluentialProspects for evidence-based policy in Russia: The public administrators’ points of view
Mikhaylova, O. V., & Batovrina, E. V. (2022) · Вестник Санкт-Петербургского университета. Серия 12. Социология · doi:10.21638/spbu12.2022.202 · Cited by 1
O ESPLENDOR E A MISÉRIA DA EPIDEMIOLOGIA NA AVALIAÇÃO EM PROMOÇÃO DA SAÚDE
Potvin, L., & Chabot, P. (2014) · doi:10.29327/561261.1-1 · Cited by 1
Bodstein, R. (2014) · doi:10.29327/561261.1-13 · Cited by 1
Jurzak-Mączka, K., Strzemecka, S., & Krzemińska, K. (2023) · Kultura i Społeczeństwo · doi:10.35757/kis.2023.67.2.8 · Cited by 1
The Role of Pediatric Nurses During Preventable Adverse Event Disclosure: A Scoping Review
Sexton, J. R., & Kelly‐Weeder, S. (2024) · Journal of Patient Safety · doi:10.1097/pts.0000000000001239 · Cited by 1
Kupiec, T. (2015) · Central European Management Journal · doi:10.7206/mba.ce.2084-3356.144 · Cited by 1
Corposanto, C. (2014) · Open Journal of Social Science Research · doi:10.12966/ojssr.03.03.2014 · Cited by 1
“it is almost impossible to reproduce them exactly, but, even if it were possible, they are so sensitive to the context that the “same combination might fail” (Pawson, 2002).”
BackgroundArandjelović, O. (2023) · Social Sciences · doi:10.3390/socsci12030173 · Cited by 1
Reading Complexity in Social Policy Contexts: The Value of Q Methodology
Hutchinson, I. A. (2012) · Open Access Victoria University of Wellington | Te Herenga Waka (Figshare) · doi:10.26686/wgtn.16997323 · Cited by 1
“…equates reliable evidential knowledge with rigorous systematic research and scientific objective knowledge, to the neglect of peoples‟ common capability to make sense of complex situations in order to act (Marston & Watts, 2003; Mulgan, 2003b; Parsons, 2002; Pawson, 2002; UK Cabinet Office, 1999b).”
BackgroundMethodInfluentialLegislative Program Evaluation Conducted by State Legislatures in the United States
Risley, J. S. (2008) · ScholarWorks - WMU (Western Michigan University) · Cited by 1
McKillop, A. (2010) · UTS ePRESS (University of Technology Sydney) · Cited by 1
Towards a New Theory of VET Policy Evaluation and Engagement: A Multiple Stakeholder Approach
Parkes, L. E. (2012) · doi:10.26686/wgtn.16999027 · Cited by 1
“Additionally – perhaps due to capacity limitations – there has been a tendency to rely on meta-analyses of research done in other countries, and to extrapolate similarities and differences to the New Zealand context; this however runs the risk of the findings having limited relevance to New Zealand, given that much of…”
BackgroundInternalized Self-Evaluation: An Effective and Sustainable Approach to Managing for ResultsPreprint
Sharp, C. A. (2006) · SSRN Electronic Journal · doi:10.2139/ssrn.897043 · Cited by 1
“This level assumes the subordinate levels of evaluation capability, but now focuses on a more sophisticated and systematic approach to evidence-based policy or "evidence-based practice" such as in "realistic evaluation" (Kazi, 2003; Pawson, 2002a, 2002b; Pawson & Tilley, 1997, 2001; Tilley, 2000).”
BackgroundCollaboration in planning sustainable developments
Jackson, J. (2009) · Proceedings of the Institution of Civil Engineers - Management Procurement and Law · doi:10.1680/mpal.2009.162.2.69 · Cited by 1
Global restructuring and local anti-poverty action: learning from European experimental programms
Henriques, J. M. (2006) · Repositório do ISCTE-IUL · Cited by 1
The Effects of Open-Plan Offices on Employee Collaboration: A Rapid Evidence ReviewPreprint
Cripps, K. G. H. (2016) · SSRN Electronic Journal · doi:10.2139/ssrn.2728676 · Cited by 1
Evaluating the Effectiveness of Rural Development Policy in Meeting Environmental Objectives
Yang, A. (2014) · ERA · doi:10.13140/rg.2.1.3827.3121 · Cited by 1
Leighton, D. T. H. (2017) · The University of Bath Online Publications Store (The University of Bath) · Cited by 1
Sharda, L. (2019) · White Rose eTheses Online (University of Leeds, The University of Sheffield, University of York) · Cited by 1
Collaborative group reasoning in ward rounds: A critical realist case study
Perversi, P. (2019) · Deakin Research Online (Deakin University) · Cited by 1
Zwierzchowska-Dod, C. (2022) · Cronfa (Swansea University) · doi:10.23889/suthesis.60121 · Cited by 1
Evaluating criminal justice interventions in the field of domestic violence: a realist approach
Taylor, H. A. (2014) · Cited by 1
Pointing Towards Policy Success?: Water Policy Indicators in PracticePreprint
Cuenca, J. S., & Howlett, M. P. (2015) · SSRN Electronic Journal · doi:10.2139/ssrn.2568468 · Cited by 1
“But what kinds of knowledge or evidence are used and by whom in this process is unclear (Feldman 1989; Pawson 2002; Sanderson 2009, Nutley et al 2007).”
Powell, C. (2012) · Figshare · Cited by 1
The role of economic analysis in the decision-making process of Independent Regulatory Agencies
Schrefler, L. (2011) · UPT. Syiah Kuala University Library (Syiah Kuala University) · Cited by 1
(Mis)trusting Health Research Synthesis Studies: Exploring Transformations of 'Evidence'
Petrova, M. (2014) · Open Research Exeter (University of Exeter) · Cited by 1
Huttunen‐Lenz, M. (2012) · UEA Digital Repository (University of East Anglia)
Foreman, C. (2000) · doi:10.15760/etd.2222
“Pawson’s (2002) realist synthesis model is a similar”
BackgroundMethodInfluentialAvaliação em programas experimentais: perspectivas da «avaliação realista»
Henriques, J. M. (2003) · Cadernos de Estudos Africanos · doi:10.4000/cea.1569
INTRODUCTION: International Transferability of the Practice of Evidence-Based Crime Policies
Marshall, I. H. (2005) · European Journal on Criminal Policy and Research · doi:10.1007/s10610-005-9002-z
Special Issue on Research Methods and Methodology: Editorial
Robinson, D., & Wells, P. (2009) · People Place and Policy Online · doi:10.3351/ppp.0003.0002.0001
eGoverment for You (EGOV4U): a realist evaluation of success and failure
Horrocks, I. (2013)
Transnational Learning and Environmental Policy Advice: Theory Vs. PracticePreprint
Howlett, M. P., & Joshi‐Koop, S. (2009) · SSRN Electronic Journal · doi:10.2139/ssrn.1464667
Social science, evidence, and social policy : the role of ideology, and the importance of history
Bateman, R. B. (2010) · doi:10.14288/1.0071038
“…perspective of policy makers, rather than a reasoned discussion and review of the evidence” (Culhane et al. 1999, 4-13), policy that can suffer from “frequent lapses into crowd pleasing, political pandering, window dressing and god-acting” on the part of policy-makers (Pawson 2002 quoted in O!”
BackgroundEvans, E. (2016) · Nottingham ePrints (University of Nottingham)
Sager, F., Hadorn, S., & Mavrot, C. (2015) · BORIS (University Library Bern)
Early findings from the National Evaluation of Offending Behaviour Pathfinder Programmes.
Hatcher, R. M., & Bilby, C. A. L. (2004) · Leicester Research Archive (University of Leicester)
Reflections of evidence: locating the role of data in drug policy discourse
Adams, K. M. (2014) · Minerva Access (University of Melbourne)
59 Arts-based palliative care training, education and staff development: a scoping review
Turton, B. M., Williams, S., Burton, C. R., & Williams, L. (2017) · BMJ Supportive & Palliative Care · doi:10.1136/bmjspcare-2017-001407.59
Special issue on innovation and societal transformation – what changes when the ‘social’ comes in?
Krlev, G., Mildenberger, G., & Anheier, H. K. (2017) · International Review of Applied Economics · doi:10.1080/02692171.2017.1377409
“Social change in advanced economies is currently discussed with a preference for evidence-based policies (Pawson 2002), marked by a focus on incremental improvements on the basis of self-regulation (European Commission 2012).”
BackgroundEvaluating local partnership incentive policies : roadmap of a research project
Sorrentino, M., Spano, A., & Bellò, B. (2011) · Archivio Istituzionale della Ricerca (Universita Degli Studi Di Milano)
Review: Trial and Error in Criminal Justice Reform: Learning from Failure
Fox, C. (2011) · Probation Journal · doi:10.1177/02645505110580020603
Ten Global Trends in Regulation: A Future Outlook
van der Heijden, J., & Hodge, G. A. (2020) · Springer eBooks · doi:10.1007/978-3-030-03008-7_2-2
Dooley, S. (2020) · Arrow@dit (Dublin Institute of Technology)
Nomaini, F., Thamrin, M. H., & Bafadhal, O. M. (2021) · PengabdianMu Jurnal Ilmiah Pengabdian kepada Masyarakat · doi:10.33084/pengabdianmu.v6i4.1849
Hammervold, U. E. (2021) · University of Stavanger eBooks · doi:10.31265/usps.93
Gędek, W. (2021) · Polityka Społeczna · doi:10.5604/01.3001.0015.3614
Hoe weet je wat werkt? Over de zoektocht naar bruikbare kennis voor beleidseffectiviteit
van Eersel, G. G. (2021) · Beleidsonderzoek Online · doi:10.5553/bo/221335502021007
Nota de Apresentação dos Editores
(2021) · Boletim de Análise Político-Institucional · doi:10.38116/bapi24notaedi
The Potential of System Dynamics to Model Patient-Aided Healthcare
(2020) · Advances in healthcare information systems and administration book series · doi:10.4018/978-1-7998-2653-8.ch005
Reading Complexity in Social Policy Contexts: The Value of Q Methodology
Hutchinson, I. A. (2012) · doi:10.26686/wgtn.16997323.v1
O ensino de medicinas alternativas e complementares em escolas medicas
Christensen, M. C. (2008) · doi:10.47749/t/unicamp.2008.423046
Wong, E., Glen, J., & Williams, S. (2019) · doi:10.21203/rs.2.18919/v1
“Whilst some WAMS are led and delivered primarily by a dedicated team of university staff [8, 30-31, 35-36], others are led by medical student volunteers with supervision from university staff [13, 31, 33-34].”
BackgroundMethodTen Global Trends in Regulation: A Future Outlook
van der Heijden, J., & Hodge, G. A. (2019) · Springer eBooks · doi:10.1007/978-3-030-03008-7_2-1
“Their work follows a long tradition of scholarship that is interested in experimentalism in policy-design (Campbell, 1969), evaluating the effectiveness of policy inititiatives (Hodge, 2010), learning policy lessons (Rose, 1993) and in evidence-based policymaking (Pawson, 2002).”
BackgroundGreenhalgh, T., Robert, G., Bate, P., Macfarlane, F., & Kyriakidou, O. (2005) · Wiley eBooks · doi:10.1002/9780470987407.refs
A UTILIZAÇÃO DE EVIDÊNCIAS NA CONSTRUÇÃO E NA EXPANSÃO DA REDE PERNAMBUCANA DE MUNICÍPIOS SAUDÁVEIS
de Sá, R. M. P. F., Schmaller, V. P. V., SALLES, R., & FREIRE, S. (2014) · doi:10.29327/561261.1-12
Potvin, L. (2014) · doi:10.29327/561261.1-20
Figueiró, A. C., Samico, I., Felisberto, E., & Hartz, Z. (2014) · doi:10.29327/561261.1-17
PROMOÇÃO DA SAÚDE E O SISTEMA NACIONAL DE SAÚDE NO BRASIL: UM ALINHAMENTO CONCEITUAL NECESSÁRIO
de Andrade, L. O. M., & Barreto, I. C. D. H. C. (2014) · doi:10.29327/561261.1-7
DA NATUREZA DOS PROGRAMAS DE PROMOÇÃO DA SAÚDE COMO AÇÕES
Potvin, L. (2014) · doi:10.29327/561261.1-8
de Sá, R. M. P. F., SALLES, R., FREIRE, S., & Schmaller, V. P. V. (2014) · doi:10.29327/561261.1-19
CONSTRUINDO PARCERIAS EM PROJETOS DE ATENÇÃO PRIMÁRIA À SAÚDE
Medina, M. G., Nunes, C. A., AQUINO, R., OLIVEIRA, S., & Vilasbôas, A. L. (2014) · doi:10.29327/561261.1-18
La metodología de evaluación en AEVAL
Callado, J. R. (2022) · Gestión y Análisis de Políticas Públicas · doi:10.24965/gapp.11011
Szifris, K. (2021) · Bristol University Press eBooks · doi:10.51952/9781529205565.bm002
Analisi dell'impatto sociale del volontariato organizzato. Il caso del CAV di Pescara
Accorinti, M., & Cerbara, L. (2023) · WELFARE E ERGONOMIA · doi:10.3280/we2022-002003
Analisis Bibliometrik Kebijakan Berbasis Bukti dalam Bidang Pendidikan
Febriano, R. D., & Yuadi, I. (2023) · CAKRAWALA · doi:10.32781/cakrawala.v17i2.552
Exploring young people’s constructions of domestic violence as portrayed in the media.
Mtshali, Z. P. (2022) · ResearchSpace (University of KwaZulu-Natal) · doi:10.29086/10413/22827
Sharda, L., Yasir, M., & Robinson, L. (2024) · Journal of Forensic Psychiatry and Psychology · doi:10.1080/14789949.2024.2350506
“Using the ‘descriptive analytical method’ in which all studies are charted against a common framework (Pawson, 2002) the data related to the barriers and facilitators of patient as educator programmes were extracted and tabulated.”
MethodEvidence-Informed Public Health Policy
Baum, F. E., & Popay, J. (2024) · Elsevier eBooks · doi:10.1016/b978-0-323-99967-0.00094-6
Life After Stroke and Supportive Stroke Pathways: Protocol For A Rapid Realist Review
Lennon, O. C., O'Neill, M., Walsh, K. R., & Horgan, F. (2025) · HRB Open Research · doi:10.12688/hrbopenres.14034.1
Life After Stroke and Supportive Stroke Pathways: Protocol For A Rapid Realist Review
Lennon, O. C., O'Neill, M., Walsh, K. R., & Horgan, F. (2025) · HRB Open Research · doi:10.12688/hrbopenres.14034.2
Lennon, O. C., O'Neill, M., Walsh, K. R., & Horgan, F. (2025) · HRB Open Research · doi:10.12688/hrbopenres.14034.3
Carolan, M. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5396164
Sustainability in Healthcare: The Role of Digital Technologies for Improving Patient Engagement
Schiavone, F., Zeuli, F., Grieva, O., Escobar, O., & Bernhard, F. (2025) · Business Ethics the Environment & Responsibility · doi:10.1111/beer.70035
A scoping review of risk and protective factors for right-wing extremist violence
Knäble, J., Breiling, L., Brodführer, A., Sadowski, F., Heil, L., & Rettenberger, M. (2025) · Aggression and Violent Behavior · doi:10.1016/j.avb.2025.102098
Translanguaging in content-based school assessments: a scoping review
Kelly, R., Ryschka, D., & Dooley, K. (2025) · International Journal of Bilingual Education and Bilingualism · doi:10.1080/13670050.2025.2587726
“Such detail enables readers of research evidence reviews to make decisions (Pawson 2002).”
Mrklas, K., Boyd, J. M., Shergill, S., Merali, S., Khan, M., Moser, C., Nowell, L. S., Goertzen, A., Swain, L. A., Pfadenhauer, L. M., et al. (2024) · Figshare · doi:10.6084/m9.figshare.26657206
Mrklas, K., Boyd, J. M., Shergill, S., Merali, S., Khan, M., Moser, C., Nowell, L. S., Goertzen, A., Swain, L. A., Pfadenhauer, L. M., et al. (2024) · Figshare · doi:10.6084/m9.figshare.26657209
Mrklas, K., Boyd, J. M., Shergill, S., Merali, S., Khan, M., Nowell, L. S., Goertzen, A., Pfadenhauer, L. M., Paúl, K., Sibley, K. M., et al. (2024) · Figshare · doi:10.6084/m9.figshare.26559371
Mead, E. S., Teeling, S. P., & McNamara, M. S. (2026) · Sustainability · doi:10.3390/su18062942 · Cited by 1
van Santvoord, V. D. (2026) · Journal of Economic Criminology · doi:10.1016/j.jeconc.2026.100218
Beal, E. M., Lipinski, D., & Zarotti, N. (2026) · The Clinical Neuropsychologist · doi:10.1080/13854046.2026.2694493
Low, J. L., Low, J. L., Vijayaraj, E., & Sevdalis, N. (2026) · Research Square · doi:10.21203/rs.3.rs-10465484/v1
Santos, C. R. B. (2010) · Americanae (AECID Library)
Metadata
- Authors
- Pawson, R.
- Affiliations
- Queen Mary University of London (GB)
- Corresponding author
- Pawson, Ray
- ORCID
- Pawson, Ray
- Venue
- Evaluation
- Publisher
- SAGE Publications
- Published
- 2002-04
- Volume
- 8
- Issue
- 2
- Pages
- 157-181
- ISSN
- 1356-3890 (ISSN-L)
- 1461-7153
- OpenAlex
- W2073020593
- Semantic Scholar
- 0a46f8e2fe3872100f441ff87ecd471f3fcf7e09
- Cited by
- 403
- References
- 26
- Citations by year
- 2026: 7 · 2025: 16 · 2024: 19 · 2023: 25 · 2022: 18 · 2021: 28 · 2020: 20 · 2019: 18 · 2018: 11 · 2017: 16 · 2016: 11 · 2015: 21 · 2014: 30 · 2013: 18 · 2012: 22
- FWCI
- 17.56
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Evaluation (publishedVersion)
- OpenGrey (Institut de l'Information Scientifique et Technique) (submittedVersion)
- Topics
- Social Sciences › Public Administration › Evaluation and Performance Assessment
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Health Sciences › Health Professions › General Health Professions › Health Policy Implementation Science
- Keywords
- evidence-based policy, systematic reviews, meta-analysis, realist synthesis, policy cycle, science-policy interface, policy decision-making
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review