A Systematic Review of Theory-Driven Evaluation Practice From 1990 to 2009

Coryn, Chris L. S.; Noakes, Lindsay A.; Westine, Carl D.; Schröter, Daniela C.

doi:10.1177/1098214010389321

Cited by 531Evaluation and Performance AssessmentAdvanced Causal Inference TechniquesPublic Policy and Administration Research

A Systematic Review of Theory-Driven Evaluation Practice From 1990 to 2009

Coryn, Chris L. S., Noakes, Lindsay A., Westine, Carl D., Schröter, Daniela C.·2011·American Journal of Evaluation

DOI 10.1177/1098214010389321

Open this paper in CoChat — free account

Abstract

Although the general conceptual basis appeared far earlier, theory-driven evaluation came to prominence only a few decades ago with the appearance of Chen’s 1990 book Theory-Driven Evaluations. Since that time, the approach has attracted many supporters as well as detractors. In this paper, 45 cases of theory-driven evaluations, published over a twenty-year period, are systematically examined to ascertain how closely theory-driven evaluation practices comport with the key tenants of theory-driven evaluation as described and prescribed by prominent theoretical writers. Evidence derived from this review to repudiate or substantiate many of the claims put forth both by critics of and advocates for theory-driven forms of evaluation are presented and an agenda for future research on the approach is recommended.

Integrity signals

Registry

Crossref · 10.1177/1098214010389321

Cites

167 of 167 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Roughly half (21/45) of the theory-driven evaluations found by Coryn et al. were evaluations of health interventions.
“Roughly half (21/45) were evaluations of health interventions [4].”
From Using theory of change to design and evaluate public health interventions: a systematic review (2015) · cites this paper as [4]
Coryn et al. found 45 examples of theory-driven evaluation in the peer-reviewed literature between 1990 and 2009.
“They found 45 examples of theory-driven evaluation in the peer-reviewed literature between 1990 and 2009.”
From Using theory of change to design and evaluate public health interventions: a systematic review (2015) · cites this paper as [4]
It was difficult to make an assessment of quality of the included papers.
“We did extract data on a checklist of ToC components proposed by Vogel [8] and principles of theory-driven evaluation by Coryn et al. [4], but it was difficult to make an assessment of quality.”
From Using theory of change to design and evaluate public health interventions: a systematic review (2015) · cites this paper as [4]
Coryn et al. report that programme theory was not used in any meaningful way to develop evaluation questions or plan and conduct and interpret the analysis.
“This is similar to the conclusions drawn by Coryn et al. who report that programme theory was not used in any meaningful way to develop evaluation questions or plan and conduct and interpret the analysis [4].”
From Using theory of change to design and evaluate public health interventions: a systematic review (2015) · cites this paper as [4]
Coryn et al. found only three papers describing theory-driven evaluation of health interventions using ToC.
“given that Coryn et al. [4] found only three papers describing theory-driven evaluation of health interventions using ToC.”
From Using theory of change to design and evaluate public health interventions: a systematic review (2015) · cites this paper as [4]

Cochrane Handbook for Systematic Reviews of Interventions: Cochrane Book Series
(2008) · doi:10.1002/9780470712184 · Cited by 25,048 · see citing works
Introduction to Meta‐Analysis
Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. R. (2009) · doi:10.1002/9780470743386 · Cited by 14,225 · see citing works
Untitled work
(2002) · Cited by 13,211 · see citing works
Introduction to Meta‐Analysis
Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. R. (2021) · doi:10.1002/9781119558378 · Cited by 9,799 · see citing works
Book Reviews: Kerlinger, Fred N. Foundations of Behavioral Research. (2nd ed.) New York: Holt, Rinehart and Winston, 1973. 741 + xx pp. $13.95
Stockton, R. A., & Kerlinger, F. N. (1974) · American Educational Research Journal · doi:10.2307/1162206 · Cited by 7,356 · see citing works
Evaluation: A Systematic Approach
Rossi, P. H., Lipsey, M. W., & Freeman, H. E. (1979) · Cited by 2,943 · see citing works
Untitled work
(1966) · Social Science Education Consortium eBooks · Cited by 2,483 · see citing works
Utilization-Focused Evaluation
(2018) · The SAGE Encyclopedia of Educational Research, Measurement, and Evaluation · doi:10.4135/9781506326139.n729 · Cited by 2,315 · see citing works
Evaluation: A Systematic Approach
Wallace, D. P. (2005) · Library & Information Science Research · doi:10.1016/j.lisr.2004.09.009 · Cited by 1,938 · see citing works
Developmental Evaluation: Applying Complexity Concepts to Enhance Innovation and Use
Patton, M. Q. (2010) · Cited by 1,740 · see citing works
Utilization-focused evaluation : the new century text
Patton, M. Q. (1997) · Cited by 1,469 · see citing works
Standards of Evidence: Criteria for Efficacy, Effectiveness and Dissemination
Flay, B. R., Biglan, A., Boruch, R. F., Castro, F. G., Gottfredson, D., Kellam, S., Mościcki, E. K., Schinke, S., Valentine, J. C., & Ji, P. (2005) · Prevention Science · doi:10.1007/s11121-005-5553-y · Cited by 1,338 · see citing works
Synthesizing Research: A Guide for Literature Reviews
Cooper, H. (1998) · Cited by 1,285 · see citing works
Theory-driven evaluations
Chen, H.-T. (1990) · Medical Entomology and Zoology · Cited by 1,227 · see citing works
Using Programme Theory to Evaluate Complicated and Complex Aspects of Interventions
Rogers, P. J. (2008) · Evaluation · doi:10.1177/1356389007084674 · Cited by 999 · see citing works
Evaluation thesaurus, 4th ed.
Scriven, M. (1991) · Cited by 954 · see citing works
Evaluation: A Systematic Approach.
Patton, M. Q., Rossi, P. H., Freeman, H. E., & Wright, S. R. (1981) · Contemporary Sociology A Journal of Reviews · doi:10.2307/2066947 · Cited by 933 · see citing works
Evaluation : methods for studying programs and policies
Weiss, C. H. (1997) · Cited by 931 · see citing works
Logic models: a tool for telling your programs performance story
McLaughlin, J. A., & Jordan, G. (1999) · Evaluation and Program Planning · doi:10.1016/s0149-7189(98)00042-1 · Cited by 740 · see citing works
Framing participatory evaluation
Cousins, J. B., & Whitmore, E. (1998) · New Directions for Evaluation · doi:10.1002/ev.1114 · Cited by 691 · see citing works
Theory‐based evaluation: Past, present, and future
Weiss, C. H. (1997) · New Directions for Evaluation · doi:10.1002/ev.1086 · Cited by 634 · see citing works
Evaluation Theory, Models, and Applications
Stufflebeam, D. L., & Coryn, C. L. S. (2007) · Cited by 631 · see citing works
Self-citation
Journal of Applied Sociology
University of Southern California., & Southern California Sociological Society. (2014) · DOAJ (DOAJ: Directory of Open Access Journals) · Cited by 614 · see citing works
Foundations of Program Evaluation: Theories of Practice.
Chen, H.-T., Shadish, W. R., Cook, T. D., & Leviton, L. C. (1992) · Contemporary Sociology A Journal of Reviews · doi:10.2307/2074800 · Cited by 534 · see citing works
The Case for Participatory Evaluation
Cousins, J. B., & Earl, L. M. (1992) · Educational Evaluation and Policy Analysis · doi:10.3102/01623737014004397 · Cited by 491 · see citing works
How Can Theory-Based Evaluation Make Greater Headway?
Weiss, C. H. (1997) · Evaluation Review · doi:10.1177/0193841x9702100405 · Cited by 484 · see citing works
Foundations of program evaluation. Theories of practice
Bond, J. J. (1992) · International Journal of Nursing Studies · doi:10.1016/0020-7489(92)90034-e · Cited by 456 · see citing works
Evaluating With Sense
Chen, H.-T., & Rossi, P. H. (1983) · Evaluation Review · doi:10.1177/0193841x8300700301 · Cited by 432 · see citing works
Theory-based impact evaluation: principles and practice
White, H. (2009) · Journal of Development Effectiveness · doi:10.1080/19439340903114628 · Cited by 427 · see citing works
Is a Residential Relocation a Good Opportunity to Change People’s Travel Behavior? Results From a Theory-Driven Intervention Study
Bamberg, S. (2006) · Environment and Behavior · doi:10.1177/0013916505285091 · Cited by 394 · see citing works
Practical program evaluation : assessing and improving planning, implementation, and effectiveness
Chen, H.-T. (2005) · Cited by 382 · see citing works
Evaluation Methodology Basics: The Nuts and Bolts of Sound Evaluation
Davidson, E. J. (2005) · Sage eBooks · doi:10.4135/9781452230115 · Cited by 365 · see citing works
Practical Program Evaluation
Chen, H.-T. (2005) · doi:10.4135/9781412985444 · Cited by 365 · see citing works
Utilization-focused Evaluation
Patton, M. Q. (2005) · Kluwer Academic Publishers eBooks · doi:10.1007/0-306-47559-6_23 · Cited by 362 · see citing works
Theory-Based Evaluation and Types of Complexity
Stame, N. (2004) · Evaluation · doi:10.1177/1356389004043135 · Cited by 350 · see citing works
Current Empirical Research on Evaluation Utilization
Cousins, J. B., & Leithwood, K. A. (1986) · Review of Educational Research · doi:10.3102/00346543056003331 · Cited by 338 · see citing works
Untitled work
(1998) · Cited by 326 · see citing works
The Logic Model Guidebook: Better Strategies for Great Results
Knowlon, L. W., & Philips, C. C. (2008) · Medical Entomology and Zoology · Cited by 323 · see citing works
Theory as method: small theories of treatments
Lipsey, M. W. (1993) · New Directions for Program Evaluation · doi:10.1002/ev.1637 · Cited by 322 · see citing works
Evaluation--promise and performance
Wholey, J. S. (1979) · Cited by 303 · see citing works
Research on Evaluation Use
Johnson, K., Greenseid, L. O., Toal, S. A., et al. (2009) · American Journal of Evaluation · doi:10.1177/1098214009341660 · Cited by 275 · see citing works
Program theory evaluation: Practice, promise, and problems
Rogers, P. J., Petrosino, A., Huebner, T. A., & Hacsi, T. A. (2000) · New Directions for Evaluation · doi:10.1002/ev.1177 · Cited by 274 · see citing works
Using program theory in evaluation
Bickman, L. (1987) · Jossey-Bass eBooks · Cited by 265 · see citing works
Measuring Program Outcomes: A Practical Approach
Roberts, N. S. (1998) · Journal of Park and Recreation Administration · Cited by 262 · see citing works
Reconstructing Program Theories: Methods Available and Problems to be Solved
Leeuw, F. L. (2003) · American Journal of Evaluation · doi:10.1016/s1098-2140(02)00271-0 · Cited by 254 · see citing works
Evaluation use: Theory, research, and practice since 1986
Shulha, L. M., & Cousins, J. B. (1997) · Evaluation Practice · doi:10.1016/s0886-1633(97)90027-1 · Cited by 245 · see citing works
The program logic model as an integrative framework for a multimethod evaluation
Cooksy, L. J., Gill, P., & Kelly, P. A. (2001) · Evaluation and Program Planning · doi:10.1016/s0149-7189(01)00003-9 · Cited by 238 · see citing works
Systematic Evaluation
Stufflebeam, D. L., & Shinkfield, A. J. (1985) · Springer eBooks · doi:10.1007/978-94-009-5656-8 · Cited by 236 · see citing works
Establishing Essential Competencies for Program Evaluators
Stevahn, L., King, J. A., Ghere, G. S., & Minnema, J. E. (2005) · American Journal of Evaluation · doi:10.1177/1098214004273180 · Cited by 230 · see citing works
The Program Evaluation Standards
(2005) · Encyclopedia of Evaluation · doi:10.4135/9781412950558.n446 · Cited by 224 · see citing works
Comer's School Development Program in Chicago: A Theory-Based Evaluation
Cook, T. D., Murphy, R. F., & Hunt, H. D. (2000) · American Educational Research Journal · doi:10.3102/00028312037002535 · Cited by 220 · see citing works
The Multi-Goal, Theory-Driven Approach to Evaluation: A Model Linking Basic and Applied Social Science
Chen, H.-T., & Rossi, P. H. (1980) · Social Forces · doi:10.1093/sf/59.1.106 · Cited by 219 · see citing works
Theory-Based Evaluation in Practice
Birckmayer, J. D., & Weiss, C. H. (2000) · Evaluation Review · doi:10.1177/0193841x0002400404 · Cited by 217 · see citing works
Testing the generalizability of intervening mechanism theories: Understanding the effects of adolescent drug use prevention interventions
Donaldson, S. I., Graham, J. W., & Hansen, W. B. (1994) · Journal of Behavioral Medicine · doi:10.1007/bf01858105 · Cited by 211 · see citing works
Theory‐Driven Subgroup‐Specific Evaluation of an Intervention to Reduce Private Car Use1
Bamberg, S., & Schmidt, P. (2001) · Journal of Applied Social Psychology · doi:10.1111/j.1559-1816.2001.tb02675.x · Cited by 208 · see citing works
The theory-driven approach to validity
Chen, H.-T., & Rossi, P. H. (1987) · Evaluation and Program Planning · doi:10.1016/0149-7189(87)90025-5 · Cited by 204 · see citing works
Program Theory-Driven Evaluation Science
Donaldson, S. I. (2007) · Routledge eBooks · doi:10.4324/9780203809730 · Cited by 201 · see citing works
Evaluation Theory, Models, & Applications
Wiersma, W. (2008) · DOAJ (DOAJ: Directory of Open Access Journals) · doi:10.56645/jmde.v6i11.197 · Cited by 196 · see citing works
Logic Modeling Methods in Program Evaluation
Schröter, D. C., & Urschel, J. L. (2008) · Journal of MultiDisciplinary Evaluation · doi:10.56645/jmde.v5i9.170 · Cited by 176 · see citing works
Self-citation
Reconstructing Program Theories: Methods Available and Problems to be Solved
Leeuw, F. L. (2003) · American Journal of Evaluation · doi:10.1177/109821400302400102 · Cited by 170 · see citing works
Linking empirically based theory and evaluation: The family bereavement program
Sandler, I. N., West, S. G., Baca, L. M., Pillow, D. R., Gersten, J. C., Rogosch, F. A., Virdin, L. M., Beals, J., Reynolds, K. D., Kallgren, C. A., et al. (1992) · American Journal of Community Psychology · doi:10.1007/bf00937756 · Cited by 162 · see citing works
What Guides Evaluation? A Study of How Evaluation Practice Maps onto Evaluation Theory
Christie, C. A. (2003) · New Directions for Evaluation · doi:10.1002/ev.72 · Cited by 159 · see citing works
Theory‐based evaluation: Reflections ten years on: Theory‐based evaluation: Past, present, and future
Rogers, P., & Weiss, C. H. (2007) · New Directions for Evaluation · doi:10.1002/ev.225 · Cited by 150 · see citing works
Evaluation Models: New Directions for Evaluation
Stufflebeam, D. L. (2001) · Cited by 148 · see citing works
The Evaluation of Large Research Initiatives
Trochim, W. M. K., Marcus, S. E., Mâsse, L. C., Moser, R. P., & Weld, P. C. (2008) · American Journal of Evaluation · doi:10.1177/1098214007309280 · Cited by 144 · see citing works
Taking Stock of Empowerment Evaluation
Miller, R. L., & Campbell, R. (2006) · American Journal of Evaluation · doi:10.1177/109821400602700303 · Cited by 140 · see citing works
Comer's School Development Program in Prince George's County, Maryland: A Theory-Based Evaluation
Cook, T. D., Habib, F.-N., Phillips, M., Settersten, R. A., Shagle, S. C., & Değirmencioğlu, S. M. (1999) · American Educational Research Journal · doi:10.3102/00028312036003543 · Cited by 140 · see citing works
Which links in which theories shall we evaluate?
Weiss, C. H. (2000) · New Directions for Evaluation · doi:10.1002/ev.1180 · Cited by 138 · see citing works
Issues in the theory-driven perspective
Chen, H.-T., & Rossi, P. H. (1989) · Evaluation and Program Planning · doi:10.1016/0149-7189(89)90046-3 · Cited by 132 · see citing works
New Directions in Program Evaluation
(1986) · Evaluation Practice · doi:10.1177/109821408600700210 · Cited by 129 · see citing works
A Framework Featuring Steps and Standards for Program Evaluation
Milstein, B., Wetterhall, S., & CDC Evaluation Working Group (2000) · Health Promotion Practice · doi:10.1177/152483990000100304 · Cited by 128 · see citing works
New Directions for Evaluation
Coryn, C. L. S. (2005) · Journal of MultiDisciplinary Evaluation · doi:10.56645/jmde.v2i3.114 · Cited by 128 · see citing works
Self-citation
Roles for Theory in Contemporary Evaluation Practice: Developing Practical Knowledge
Donaldson, S. I., & Lipsey, M. W. (2006) · SAGE Publications Ltd eBooks · doi:10.4135/9781848608078.n2 · Cited by 126 · see citing works
Logic Modeling Methods in Program Evaluation
Frechtling, J. A. (2007) · Medical Entomology and Zoology · Cited by 122 · see citing works
Books, Buildings, and Learning Outcomes: An Impact Evaluation of World Bank Support to Basic Education in Ghana
White, H. N. (2004) · Cited by 121 · see citing works
CHANGING TRAVEL-MODE CHOICE AS RATIONAL CHOICE:
Bamberg, S., & Schmidt, P. (1998) · Rationality and Society · doi:10.1177/104346398010002005 · Cited by 118 · see citing works
Unpacking the Participatory Process
Weaver, L. M., & Cousins, J. B. (2004) · Journal of MultiDisciplinary Evaluation · doi:10.56645/jmde.v1i1.144 · Cited by 117 · see citing works
Evaluating Social Programs and Problems
Donaldson, S. I., & Scriven, M. (2003) · Routledge eBooks · doi:10.4324/9781410606556 · Cited by 112 · see citing works
Evaluation Use
Fleischer, D. N., & Christie, C. A. (2009) · American Journal of Evaluation · doi:10.1177/1098214008331009 · Cited by 111 · see citing works
A Statewide Evaluation of the Effectiveness of Media Literacy Training to Prevent Tobacco Use Among Adolescents
Pinkleton, B. E., Austin, E. W., Cohen, M. A., Miller, A., & Fitzgerald, E. (2007) · Health Communication · doi:10.1080/10410230701283306 · Cited by 106 · see citing works
Toward an Agenda for Research on Evaluation
Henry, G. T., & Mark, M. M. (2003) · New Directions for Evaluation · doi:10.1002/ev.77 · Cited by 105 · see citing works
Assessing the Effectiveness of Business Support Services in England
Mole, K. F., Hart, M., Roper, S., & Saal, D. S. (2009) · International Small Business Journal Researching Entrepreneurship · doi:10.1177/0266242609338755 · Cited by 102 · see citing works
Establishing evaluative conclusions: A distinction between general and working logic
Fournier, D. M. (1995) · New Directions for Evaluation · doi:10.1002/ev.1017 · Cited by 96 · see citing works
Theory-Based Evaluation: The Case of Social Funds
Carvalho, S., & White, H. (2004) · American Journal of Evaluation · doi:10.1177/109821400402500202 · Cited by 94 · see citing works
Evaluability Assessment From 1986 to 2006
Trevisan, M. S. (2007) · American Journal of Evaluation · doi:10.1177/1098214007304589 · Cited by 94 · see citing works
Using Theory to Improve Program and Policy Evaluations.
Sabatier, P., Chen, H.-T., & Rossi, P. H. (1994) · Contemporary Sociology A Journal of Reviews · doi:10.2307/2076456 · Cited by 93 · see citing works
Semi-structured interview protocol for constructing logic models
Gugiu, P. C., & Rodríguez‐Campos, L. (2007) · Evaluation and Program Planning · doi:10.1016/j.evalprogplan.2007.08.004 · Cited by 92 · see citing works
A Review and Synthesis of Current Research on Cross-Cultural Evaluation
Chouinard, J. A., & Cousins, J. B. (2009) · American Journal of Evaluation · doi:10.1177/1098214009349865 · Cited by 91 · see citing works
Managing Fear in Public Health Campaigns: A Theory-Based Formative Evaluation Process
Cho, H., & Witte, K. (2005) · Health Promotion Practice · doi:10.1177/1524839904263912 · Cited by 83 · see citing works
Minimalist theory: The least theory that practice requires
Scriven, M. (1998) · American Journal of Evaluation · doi:10.1016/s1098-2140(99)80180-5 · Cited by 82 · see citing works
Using Theory to Improve Program and Policy Evaluations.
Freeman, D. M., Chen, H.-T., & Rossi, P. H. (1993) · Social Forces · doi:10.2307/2580149 · Cited by 76 · see citing works
Ascertaining causality in theory‐based evaluation
Davidson, E. J. (2000) · New Directions for Evaluation · doi:10.1002/ev.1178 · Cited by 74 · see citing works
Treatment Strength and Integrity: Models and Methods.
Cordray, D. S., & Pion, G. M. (2006) · American Psychological Association eBooks · doi:10.1037/11384-006 · Cited by 74 · see citing works
Theory-driven evaluation in action: lessons from a $20 million statewide Work and Health Initiative
Donaldson, S. I., & Gooler, L. E. (2003) · Evaluation and Program Planning · doi:10.1016/s0149-7189(03)00052-1 · Cited by 72 · see citing works
Reported Influence of Evaluation Data on Decision Makers’ Actions
Christie, C. A. (2007) · American Journal of Evaluation · doi:10.1177/1098214006298065 · Cited by 72 · see citing works
Using decision aids in community-based primary care: A theory-driven evaluation with ethnically diverse patients
Frosch, D. L., Légaré, F., & Mangione, C. M. (2008) · Patient Education and Counseling · doi:10.1016/j.pec.2008.07.040 · Cited by 71 · see citing works
Program Theory: Not Whether Programs Work but How They Work
Rogers, P. (2005) · Kluwer Academic Publishers eBooks · doi:10.1007/0-306-47559-6_13 · Cited by 69 · see citing works
Effects of traditional classroom and distance continuing education: a theory-driven evaluation of a vaccine-preventable diseases course
Umble, K. E., Cervero, R. M., Yang, B., & Atkinson, W. L. (2000) · American Journal of Public Health · doi:10.2105/ajph.90.8.1218 · Cited by 69 · see citing works
A Theory-Driven Intervention and Evaluation to Explore Family Caregiver Empowerment
Heflinger, C. A., Bickman, L., Northrup, D. A., et al. (1997) · Journal of Emotional and Behavioral Disorders · doi:10.1177/106342669700500306 · Cited by 67 · see citing works
The Fine Line between Evaluation and Explanation
Scriven, M. (1994) · Evaluation Practice · doi:10.1177/109821409401500108 · Cited by 66 · see citing works
The Role of Evaluation in Research—Practice Integration Working Toward the ‘‘Golden Spike’’
Urban, J. B., & Trochim, W. M. K. (2009) · American Journal of Evaluation · doi:10.1177/1098214009348327 · Cited by 64 · see citing works
Minimalist Theory: The Least Theory That Practice Requires
Scriven, M. (1998) · American Journal of Evaluation · doi:10.1177/109821409801900105 · Cited by 64 · see citing works
Evaluation Use: Theory, Research, and Practice Since 1986
Shulha, L. M., & Cousins, J. B. (1997) · Evaluation Practice · doi:10.1177/109821409701800121 · Cited by 62 · see citing works
Developing Standards for Empirical Examinations of Evaluation Theory
Miller, R. L. (2010) · American Journal of Evaluation · doi:10.1177/1098214010371819 · Cited by 61 · see citing works
Using reference databases.
Reed, J. G., & Baxter, P. M. (2009) · Cited by 59 · see citing works
The role of experience in formulating theories of evaluation practice
Chelimsky, E. (1998) · American Journal of Evaluation · doi:10.1016/s1098-2140(99)80179-9 · Cited by 58 · see citing works
A Bibliometric Analysis of the Academic Influences of and on Evaluation Theorists’ Published Works
Heberger, A. E., Christie, C. A., & Alkin, M. C. (2010) · American Journal of Evaluation · doi:10.1177/1098214009354120 · Cited by 57 · see citing works
Mediator and Moderator Analysis in Program Development
Donaldson, S. I. (2001) · Sage eBooks · doi:10.4135/9781412991445.n32 · Cited by 57 · see citing works
Improving evaluation theory through the empirical study of evaluation practice
Smith, N. L. (1993) · Evaluation Practice · doi:10.1016/0886-1633(93)90051-p · Cited by 55 · see citing works
Current Challenges in Development Evaluation
Conlin, S., & Stirrat, R. L. (2008) · Evaluation · doi:10.1177/1356389007087539 · Cited by 54 · see citing works
Theory-Driven Program Evaluation in the New Millennium
Donaldson, S. I., & Scriven, M. (2003) · Routledge eBooks · doi:10.4324/9781410606556-13 · Cited by 53 · see citing works
The political environment of evaluation and what it means for the development of the field
Chelimsky, E. (1995) · Evaluation Practice · doi:10.1016/0886-1633(95)90035-7 · Cited by 52 · see citing works
Integrating a Comparison Group Design into a Theory of Change Evaluation: The Case of the Urban Health Initiative
Weitzman, B. C., Silver, D., & Dillman, K.-N. (2002) · American Journal of Evaluation · doi:10.1177/109821400202300402 · Cited by 52 · see citing works
A quantitative analysis of the impact of a computerised information system on nurses’ clinical practice using a realistic evaluation framework
Oroviogoicoechea, C., & Watson, R. A. (2009) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2009.08.008 · Cited by 51 · see citing works
Integrating a Comparison Group Design into a Theory of Change Evaluation: The Case of the Urban Health Initiative
Weitzman, B. C., Silver, D., & Dillman, K.-N. (2002) · American Journal of Evaluation · doi:10.1016/s1098-2140(02)00236-9 · Cited by 51 · see citing works
A realist theory of evaluation practice
Mark, M. M., Henry, G. T., & Julnes, G. (1998) · New Directions for Evaluation · doi:10.1002/ev.1098 · Cited by 49 · see citing works
The Role of Experience in Formulating Theories of Evaluation Practice
Chelimsky, E. (1998) · American Journal of Evaluation · doi:10.1177/109821409801900104 · Cited by 49 · see citing works
A User-Friendly Approach to Program Evaluation and Effective Community Interventions for Families at Risk of Homelessness
Mulroy, E. A., & Lauber, H. (2004) · Social Work · doi:10.1093/sw/49.4.573 · Cited by 48 · see citing works
Assessing interventions to improve child nutrition: a theory‐based impact evaluation of the Bangladesh Integrated Nutrition Project
White, H., & Masset, E. (2006) · Journal of International Development · doi:10.1002/jid.1344 · Cited by 46 · see citing works
The Strength of the Methodological Warrants for the Findings of Research on Program Evaluation Use
Brandon, P. R., & Singh, J. M. (2009) · American Journal of Evaluation · doi:10.1177/1098214009334507 · Cited by 44 · see citing works
The palliative management of fungating malignant wounds – generalising from multiple-case study data using a system of reasoning
Grocott, P., & Cowley, S. A. (2001) · International Journal of Nursing Studies · doi:10.1016/s0020-7489(00)00098-5 · Cited by 44 · see citing works
Theory-driven evaluations: Need, difficulties, and options
Chen, H.-T. (1994) · Evaluation Practice · doi:10.1016/0886-1633(94)90063-9 · Cited by 43 · see citing works
Applying theory-driven evaluation to the British Crime Reduction Programme
Tilley, N. (2004) · Criminal Justice · doi:10.1177/1466802504048465 · Cited by 42 · see citing works
The application of program theory to the evaluation of a managed mental health care system
Bickman, L. (1996) · Evaluation and Program Planning · doi:10.1016/0149-7189(96)00002-x · Cited by 39 · see citing works
Evaluators' Decision Making
Tourmen, C. (2009) · American Journal of Evaluation · doi:10.1177/1098214008327602 · Cited by 38 · see citing works
Maintaining Momentum to 2015? An impact evaluation of interventions to improve maternal and child health and nutrition in BangladeshPreprint
White, H. N., Masset, E., Blöndal, N., & Waddington, H. (2005) · RePEc: Research Papers in Economics · Cited by 37 · see citing works
The fine line between evaluation and explanation
Scriven, M. (1994) · Evaluation Practice · doi:10.1016/0886-1633(94)90062-0 · Cited by 37 · see citing works
Whither Evaluation? That All Depends
Worthen, B. R. (2001) · American Journal of Evaluation · doi:10.1016/s1098-2140(01)00139-4 · Cited by 36 · see citing works
Rooting for Evaluation: A Cliff Notes Version of My Work
Weiss, C. H. (2004) · Sage eBooks · doi:10.4135/9781412984157.n9 · Cited by 34 · see citing works
Accountability and Performance in Charter Schools in Florida: A Theory-Based Evaluation
Crew, R. E., & Anderson, M. R. (2003) · American Journal of Evaluation · doi:10.1016/s1098-2140(03)00030-4 · Cited by 33 · see citing works
Putting Theory-Oriented Evaluation Into Practice
Hense, J. U., Kriz, W. C., & Wolfe, J. (2007) · Simulation & Gaming · doi:10.1177/1046878107308078 · Cited by 32 · see citing works
The American Journal of Evaluation
(1997) · Evaluation Practice · doi:10.1177/109821409701800301 · Cited by 32 · see citing works
Network management and environmental effectiveness: the management of end-of-life vehicles in the United Kingdom and in Sweden
Manomaivibool, P. (2008) · Journal of Cleaner Production · doi:10.1016/j.jclepro.2008.01.013 · Cited by 31 · see citing works
Evaluation of Researchers and Their Research: Toward Making the Implicit Explicit
Coryn, C. L. S. (2007) · ScholarWorks - WMU (Western Michigan University) · Cited by 30 · see citing works
Self-citation
An Application of Theory-Driven Evaluation to a Drop-In Youth Center
Mercier, C., Piat, M., Péladeau, N., & Dagenais, C. (2000) · Evaluation Review · doi:10.1177/0193841x0002400103 · Cited by 30 · see citing works
School Personnel Educating the Whole Child: Impact of Character Education on Teachers’ Self-Assessment and Student Development
Chang, F., & Muñoz, M. A. (2006) · Journal of Personnel Evaluation in Education · doi:10.1007/s11092-007-9036-5 · Cited by 28 · see citing works
Finding the Impact in a Messy Intervention: Using an Integrated Design to Evaluate a Comprehensive Citywide Health Initiative
Weitzman, B. C., Mijanovich, T., Silver, D., & Brecher, C. (2009) · American Journal of Evaluation · doi:10.1177/1098214009347555 · Cited by 27 · see citing works
Theory-based evaluation of a comprehensive Latino education initiative: An interactive evaluation approach
Nesman, T. M., Batsche, C. J., & Hernandez, M. (2007) · Evaluation and Program Planning · doi:10.1016/j.evalprogplan.2007.04.004 · Cited by 27 · see citing works
The Political Environment of Evaluation and What It Means for the Development of the Field
Chelimsky, E. (1997) · Sage eBooks · doi:10.4135/9781483348896.n3 · Cited by 27 · see citing works
Implementing a system of care: Findings from the Fort Bragg Evaluation Project
Heflinger, C. A. (1996) · The Journal of Behavioral Health Services & Research · doi:10.1007/bf02518640 · Cited by 26 · see citing works
Confirmatory Program Evaluation: Applications to Early Childhood Interventions
Reynolds, A. J. (2005) · Teachers College Record The Voice of Scholarship in Education · doi:10.1111/j.1467-9620.2005.00597.x · Cited by 26 · see citing works
Applying a programme theory framework to improve livelihood interventions integrated with HIV care and treatment programmes
Kadiyala, S., Rawat, R., Roopnaraine, T., Babirye, F., & Ochai, R. (2009) · Journal of Development Effectiveness · doi:10.1080/19439340903370469 · Cited by 21 · see citing works
A Therapeutic Summer Weekend Camp for Grieving Children: Supporting Clinical Practice Through Empirical Evaluation
Farber, M. L. Z., & Sabatino, C. A. (2007) · Child and Adolescent Social Work Journal · doi:10.1007/s10560-007-0090-0 · Cited by 20 · see citing works
Accountability and Performance in Charter Schools in Florida: A Theory-Based Evaluation
Crew, R. E., & Anderson, M. R. (2003) · American Journal of Evaluation · doi:10.1177/109821400302400204 · Cited by 19 · see citing works
Theorists' models in action
Alkin, M. C., & Christie, C. A. (2005) · Jossey-Bass eBooks · Cited by 18 · see citing works
Clarifying and expanding the application of program theory-driven evaluations
Smith, N. L. (1994) · Evaluation Practice · doi:10.1016/0886-1633(94)90064-7 · Cited by 18 · see citing works
Whither Evaluation? That All Depends
Worthen, B. R. (2001) · American Journal of Evaluation · doi:10.1177/109821400102200319 · Cited by 18 · see citing works
Program Theory Building: A Strategy for Deriving Cumulative Evaluation Knowledge
TURNBULL, B. (2002) · American Journal of Evaluation · doi:10.1016/s1098-2140(02)00201-1 · Cited by 17 · see citing works
Theory-Driven Evaluation of the Work and Health Initiative: A Focus on Winning New Jobs
Donaldson, S. I., & Gooler, L. E. (2002) · American Journal of Evaluation · doi:10.1177/109821400202300310 · Cited by 17 · see citing works
Unlearning Some of our Social Scientist Habits
Davidson, E. J. (2007) · Journal of MultiDisciplinary Evaluation · doi:10.56645/jmde.v4i8.68 · Cited by 16 · see citing works
Assessing Instructional Reform in San Diego: A Theory-Based Approach
O’Day, J., & Quick, H. E. (2009) · Journal of Education for Students Placed at Risk (JESPAR) · doi:10.1080/10824660802715346 · Cited by 14 · see citing works
The Health Action Zone Initiative: Lessons from Plymouth
Cole, M. (2003) · Local Government Studies · doi:10.1080/03003930308559381 · Cited by 14 · see citing works
When Getting It Right Matters: The Case for High-Quality Policy and Program Impact Evaluations
Henry, G. (2009) · Sage eBooks · doi:10.4135/9781412995634.d9 · Cited by 14 · see citing works
Theory-Driven Evaluation of the Work and Health Initiative: A Focus on Winning New Jobs
Donaldson, S. (2002) · American Journal of Evaluation · doi:10.1016/s1098-2140(02)00202-3 · Cited by 13 · see citing works
Evaluating Aid Impact
White, H. (2009) · Palgrave Macmillan UK eBooks · doi:10.1057/9780230595163_9 · Cited by 12 · see citing works
Program Theory Building: A Strategy for Deriving Cumulative Evaluation Knowledge
Turnbull, B. (2002) · American Journal of Evaluation · doi:10.1177/109821400202300304 · Cited by 12 · see citing works
Analyzing the Assumptions of a Policy Program
Janssens, F. J. G., & de Wolf, I. (2009) · American Journal of Evaluation · doi:10.1177/1098214009341016 · Cited by 11 · see citing works
Evaluating the Process and Outcome of a Garbage Reduction Program in Taiwan
Chen, H.-T., Wang, J. C. S., & Lin, L.-H. (1997) · Evaluation Review · doi:10.1177/0193841x9702100102 · Cited by 10 · see citing works
Analysis of Caregiver Burden in South Asian Families in the Dallas-Fort Worth Metropolitan Area: Insights for Social Practice
Gupta, R., & Pillai, V. K. (2005) · Journal of Applied Sociology · doi:10.1177/19367244052200203 · Cited by 10 · see citing works
The Development of Theory-Driven Evaluation in the Military
Williams, A. P. J. W., & Morris, J. C. (2009) · American Journal of Evaluation · doi:10.1177/1098214008329522 · Cited by 9 · see citing works
Law Enforcement Training in Southeast Asia: A Theory‐Driven Evaluation
Johnson, K., Young, L. C., Foster, J. P., et al. (2006) · Police Practice and Research · doi:10.1080/15614260600825406 · Cited by 8 · see citing works
Theory‐based evaluation of AIDS‐related knowledge, attitudes, and behavior changes
Conner, R. F., Mishra, S. I., Lewis, M. A., et al. (1990) · New Directions for Program Evaluation · doi:10.1002/ev.1548 · Cited by 7 · see citing works
Book Review: Practical Program Evaluation: Assessing and Improving Planning, Implementation, and Effectiveness
Coryn, C. L. S. (2005) · American Journal of Evaluation · doi:10.1177/1098214005278767 · Cited by 6 · see citing works
Self-citation
Development of Local Program Theory: Using Theory-Oriented Evaluation to Make a Difference
Sielbeck-Bowen, K. A. (2000) · Field Methods · doi:10.1177/1525822x0001200203 · Cited by 6 · see citing works
Book Review: Program Theory-Driven Evaluation Science: Strategies and Applications, by Stewart I. Donaldson. Mahwah, NJ: Lawrence Erlbaum, 2007
Coryn, C. L. S. (2008) · American Journal of Evaluation · doi:10.1177/1098214008314981 · Cited by 3 · see citing works
Self-citation
Introduction to the Forum: Advances in Evaluating Evaluation Theory
Smith, N. L. (2010) · American Journal of Evaluation · doi:10.1177/1098214010371821 · Cited by 3 · see citing works
A Case of Policy Evaluation Utilizing a Logical Framework
Sato, Y. (2005) · Evaluation · doi:10.1177/1356389005058483 · Cited by 3 · see citing works

Flagged first, then by citations

Sorted by citations

Theory of Change: a theory-driven approach to enhance the Medical Research Council's framework for complex interventions
De Silva, M. J., Breuer, E., Lee, L., Asher, L., Chowdhary, N., Lund, C., & Patel, V. (2014) · Trials · doi:10.1186/1745-6215-15-267 · Cited by 738 · see citing works
Is realist evaluation keeping its promise? A review of published empirical studies in the field of health systems research
Marchal, B., van Belle, S., van Olmen, J., Hoerée, T., & Kegels, G. (2012) · Evaluation · doi:10.1177/1356389012442444 · Cited by 476 · see citing works
Synthesising quantitative and qualitative evidence to inform guidelines on complex interventions: clarifying the purposes, designs and outlining some methods
Noyes, J., Booth, A., Moore, G., Flemming, K., Tunçalp, Ö., & Shakibazadeh, E. (2019) · BMJ Global Health · doi:10.1136/bmjgh-2018-000893 · Cited by 497 · see citing works
Using theory of change to design and evaluate public health interventions: a systematic review
Breuer, E., Lee, L., De Silva, M., & Lund, C. (2015) · Implementation Science · doi:10.1186/s13012-016-0422-6 · Cited by 425 · see citing works
Contribution analysis: Coming of age?
Mayne, J. (2012) · Evaluation · doi:10.1177/1356389012451663 · Cited by 387 · see citing works
Travelling models and the challenge of pragmatic contexts and practical norms: the case of maternal health
Olivier de Sardan, J.-P., Diarra, A., & Moha, M. (2017) · Health Research Policy and Systems · doi:10.1186/s12961-017-0213-9 · Cited by 167 · see citing works
Planning and evaluating mental health services in low- and middle-income countries using theory of change
Breuer, E., De Silva, M. J., Shidaye, R., Petersen, I., Nakku, J., Jordans, M. J. D., Fekadu, A., & Lund, C. (2016) · British Journal of Psychiatry · doi:10.1192/bjp.bp.114.153841 · Cited by 111 · see citing works
Using workshops to develop theories of change in five low and middle income countries: lessons from the programme for improving mental health care (PRIME)
Breuer, E., De Silva, M. J., Fekadu, A., Luitel, N. P., Murhar, V., Nakku, J., Petersen, I., & Lund, C. (2014) · International Journal of Mental Health Systems · doi:10.1186/1752-4458-8-15 · Cited by 93 · see citing works
What is Essential in Developmental Evaluation? On Integrity, Fidelity, Adultery, Abstinence, Impotence, Long-Term Commitment, Integrity, and Sensitivity in Implementing Evaluation Models
Patton, M. Q. (2016) · American Journal of Evaluation · doi:10.1177/1098214015626295 · Cited by 75 · see citing works
Communicable disease control programmes and health systems: an analytical approach to sustainability
Shigayeva, A., & Coker, R. J. (2015) · Health Policy and Planning · doi:10.1093/heapol/czu005 · Cited by 63 · see citing works

Authors: Coryn, C. L. S., Noakes, L. A., Westine, C. D., & Schröter, D. C.
Affiliations: Western Michigan University (US)
Corresponding author: Coryn, Chris L. S.
ORCID: Westine, Carl D.
Schröter, Daniela C.
Venue: American Journal of Evaluation
Publisher: SAGE Publications
Published: 2010-11-12
Volume: 32
Issue: 2
Pages: 199-226
ISSN: 1098-2140 (ISSN-L)
1557-0878
DOI: 10.1177/1098214010389321
OpenAlex: W2012207800
Cited by: 531
References: 167
Citations by year: 2026: 18 · 2025: 38 · 2024: 36 · 2023: 38 · 2022: 25 · 2021: 42 · 2020: 76 · 2019: 42 · 2018: 43 · 2017: 39 · 2016: 30 · 2015: 34 · 2014: 35 · 2013: 11 · 2012: 19
FWCI: 4.76
Type: journal article
Language: en
Open access: closed
Licence: journals.sagepub.com/page/policies/text-and-data-mining-license (tdm)
Locations: American Journal of Evaluation (publishedVersion)
Topics: Social Sciences › Decision Sciences › Management Science and Operations Research › Evaluation and Performance Assessment
Physical Sciences › Mathematics › Statistics and Probability › Advanced Causal Inference Techniques
Social Sciences › Public Administration › Public Policy and Administration Research
Keywords: theory-based evaluation, evaluation theory, assessment practices, evaluation methodology, evaluation design

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

A Systematic Review of Theory-Driven Evaluation Practice From 1990 to 2009

Abstract

Integrity signals

Claims the paper makes

Claims other papers make about this one

Cites

Cited by

Related

Record

About these checks