The DOI matches its Crossref record; 41 of the 42 references it cites were found in a registry or an index.
Using Programme Theory to Evaluate Complicated and Complex Aspects of Interventions
Abstract
This article proposes ways to use programme theory for evaluating aspects of programmes that are complicated or complex. It argues that there are useful distinctions to be drawn between aspects that are complicated and those that are complex, and provides examples of programme theory evaluations that have usefully represented and address both of these. While complexity has been defined in varied ways in previous discussions of evaluation theory and practice, this article draws on Glouberman and Zimmerman's conceptualization of the differences between what is complicated (multiple components) and what is complex (emergent). Complicated programme theory may be used to represent interventions with multiple components, multiple agencies, multiple simultaneous causal strands and/or multiple alternative causal strands. Complex programme theory may be used to represent recursive causality (with reinforcing loops), disproportionate relationships (where at critical levels, a small change can make a big difference — a `tipping point') and emergent outcomes.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
Utilization-Focused Evaluation
(2018) · The SAGE Encyclopedia of Educational Research, Measurement, and Evaluation · doi:10.4135/9781506326139.n729 · Cited by 2,308
Chen, H.-T. (1990) · Medical Entomology and Zoology · Cited by 1,227
The new dynamics of strategy: Sense-making in a complex and complicated world
Kurtz, C. F., & Snowden, D. J. (2003) · IBM Systems Journal · doi:10.1147/sj.423.0462 · Cited by 1,211
Complexity and Creativity in Organizations
Stacey, R. D. (1996) · Cited by 1,189
Evaluation : methods for studying programs and policies
Weiss, C. H. (1997) · Cited by 924
Common purpose: strengthening families and neighborhoods to rebuild America
(1998) · Choice Reviews Online · doi:10.5860/choice.35-3612 · Cited by 455
Utilization-focused evaluation
Westbrook, L. (1998) · Library & Information Science Research · doi:10.1016/s0740-8188(98)90032-9 · Cited by 374
1 Complicated and Complex Systems: What Would Successful Reform of Medicare Look Like?
Glouberman, S., & Zimmerman, B. (2004) · Changing Health Care in Canada · doi:10.3138/9781442672833-004 · Cited by 358
Mathison, S. (2005) · Sage eBooks · doi:10.4135/9781412950558 · Cited by 352
Theory-Based Evaluation and Types of Complexity
Stame, N. (2004) · Evaluation · doi:10.1177/1356389004043135 · Cited by 349
Research works printed but not linked: 27
- Arnkil, R., Paper presented at the 2002 Conference of the European Evaluation Society
- Bakewell, O., 2005, The Use and Abuse of the Logical Framework Approach
- Collaborative Institute for Research, Consulting and Learning in Evaluation (CIRCLE), 2006, Third newsletter
- Douthwaite, B., paper presented at the INRM Workshop
- Eoyang, G., 1998, Evaluation in Complex Adaptive System
- Eoyang, G., 1998, Life Sciences
- Funnell, S., 1997, Evaluation News and Comment, 6, 5
- Giuliani, R., 2002, Leadership
- Glouberman, S., 2001, Towards a New Perspective on Health Policy
- Glouberman, S., Commission on the Future of Health Care in Canada, Discussion Paper 8
- Gregory, R.J., 1995, `Accountability, Responsibility, and Corruption: Managing the “Public Production Process”'
- Kankare, I., paper presented at the European Evaluation Society conference
- Nagarajan, N., 1997, Evaluating EU Expenditure Programmes: A Guide
- 10.1177/109821409401500312 · doi:10.1177/109821409401500312 · Paper page
- Patton, M.Q., paper presented at the 2003 conference of the Australasian Evaluation Society
- Pawson, R., 1997, Realistic Evaluation
- Perrin, B., 2003, Implementing the Vision: Addressing the Challenges of Results-Based Management
- Pinnegar, S., 2006, Are Complex Programs the Best Response to Complex Policy Issues? City Futures Research Centre Issues paper. Kensington, NSW
- Potter, P., paper presented at the 2004 Conference of the European Evaluation Society
- Regine, B., Complexity Science in Human Terms: A Relational Model of Business
- Rogers, P.J., 2004, Evaluation, 232
- Sibthorpe, B., 2004, Complex Adaptive Systems: A Different Way of Thinking about Health Care Systems. A Brief Synopsis of Selected Literature for Initial Work Program — Stream 1
- Sparks, D., 2003, NSDC's Theory of Change
- Weiss, C.H. (1995) `Nothing as Practical as a Good Theory: Exploring Theory-Based Evaluation for Comprehensive Community Initiatives for Children and Families', in J. P. Connell , A. C. Kubisch , L. B. Schorr and C. H. Weiss (eds) New Approaches to Evaluating Community Initiatives, pp. 65—9. Washington, DC: Aspen Institute .
- Wholey, Joseph S., 2003, Studies, 3, 6
- W. K. Kellogg Foundation, 2004, Logic Model Development Guide
- World Bank, 2004, Pages from World Bank History: The Fight Against Riverblindness
Cited by
Guidance on how to develop complex interventions to improve health and healthcare
O"Cathain, A., Croot, L., Duncan, E., Rousseau, N., Sworn, K., Turner, K. M., Yardley, L., & Hoddinott, P. (2019) · BMJ Open · doi:10.1136/bmjopen-2019-029954 · Cited by 1,591
“A programme theory describes how a specific intervention is expected to lead to its effects and under what conditions.(14) It shows the causal pathways between the content of the intervention, intermediate outcomes and long-term goals and how these interact with contextual factors.”
BackgroundEbrahim, A., Battilana, J., & Mair, J. (2014) · Research in Organizational Behavior · doi:10.1016/j.riob.2014.09.001 · Cited by 1,264
Tailored interventions to address determinants of practice
Baker, R., Camosso-Stefinovic, J., Gillies, C., Shaw, E. J., Cheater, F., Flottorp, S., Robertson, N., Wensing, M., Fiander, M., Eccles, M. P., et al. (2015) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd005470.pub3 · Cited by 711
Lessons from Complex Interventions to Improve Health
Hawe, P. (2015) · Annual Review of Public Health · doi:10.1146/annurev-publhealth-031912-114421 · Cited by 607
A Systematic Review of Theory-Driven Evaluation Practice From 1990 to 2009
Coryn, C. L. S., Noakes, L. A., Westine, C. D., & Schröter, D. C. (2011) · American Journal of Evaluation · doi:10.1177/1098214010389321 · Cited by 531
“In recent writings, others (e.g., Chen, 2005a, 2005b, 2005c; Rogers, 2008) have advocated for more contextualized, comprehensive, ecological program theory models.2 As shown in Figure 2, such models diverge considerably from the more simplistic, linear model illustrated in Figure 1.”
BackgroundInfluential
Metadata
- Authors
- Rogers, P. J.
- Affiliations
- RMIT University (AU)
- Corresponding author
- Rogers, Patricia J.
- ORCID
- Rogers, Patricia J.
- Venue
- Evaluation
- Publisher
- SAGE Publications
- Published
- 2008-01
- Volume
- 14
- Issue
- 1
- Pages
- 29-48
- ISSN
- 1356-3890 (ISSN-L)
- 1461-7153
- OpenAlex
- W2067154398
- Semantic Scholar
- 4f6aa067c6dfb05b80e1aa778684236ac3384051
- Cited by
- 999
- References
- 42
- Citations by year
- 2026: 53 · 2025: 69 · 2024: 43 · 2023: 59 · 2022: 99 · 2021: 89 · 2020: 101 · 2019: 62 · 2018: 72 · 2017: 53 · 2016: 64 · 2015: 53 · 2014: 46 · 2013: 37 · 2012: 41
- FWCI
- 29.65
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- journals.sagepub.com/page/policies/text-and-data-mining-license (tdm)
- Locations
- Evaluation (publishedVersion)
- RMIT Research Repository (RMIT University Library) (submittedVersion)
- Figshare (submittedVersion)
- Topics
- Social Sciences › Public Administration › Evaluation and Performance Assessment
- Physical Sciences › Mathematics › Statistics and Probability › Advanced Causal Inference Techniques
- Social Sciences › Public Administration › Public Policy and Administration Research
- Keywords
- program theory, complex interventions, program evaluation, emergent outcomes, tipping points
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Using Programme Theory to Evaluate Complicated and Complex Aspects of Interventions. https://app.cochat.ai/papers/10.1177/1356389007084674
- The paper
- Rogers, P. J. (2008). Using Programme Theory to Evaluate Complicated and Complex Aspects of Interventions. Evaluation. https://doi.org/10.1177/1356389007084674
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.