The DOI matches its Crossref record; 32 of the 32 references it cites were found in a registry or an index.
Toward a Conceptual Framework for Mixed-Method Evaluation Designs
Abstract
In recent years evaluators of educational and social programs have expanded their methodological repertoire with designs that include the use of both qualitative and quantitative methods. Such practice, however, needs to be grounded in a theory that can meaningfully guide the design and implementation of mixed-method evaluations. In this study, a mixed-method conceptual framework was developed from the theoretical literature and then refined through an analysis of 57 empirical mixed-method evaluations. Five purposes for mixed-method evaluations are identified in this conceptual framework: triangulation, complementarity, development, initiation, and expansion. For each of the five purposes, a recommended design is also presented in terms of seven relevant design characteristics. These design elements encompass issues about methods, the phenomena under investigation, paradigmatic framework, and criteria for implementation. In the empirical review, common misuse of the term triangulation was apparent in evaluations that stated such a purpose but did not employ an appropriate design. In addition, relatively few evaluations in this review integrated the different method types at the level of data analysis. Strategies for integrated data analysis are among the issues identified as priorities for further mixed-method work.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“The use of a mixed method approach was deemed important to achieve the primary benefits of development and expansion of the checklist (Greene,Caracelli,&Graham,1989).”
From Developing an English Language Textbook Evaluative Checklist (2013) · cites this paper as Greene,Caracelli,&Graham,1989
In the citing paper
“Ironically, in a comprehensive review of mixed-method evaluation studies, Greene and Caracelli (1989) found that methodological triangulation was actually quite rare in mixed-method research, used by only 3 of 57 studies.”
From Revisiting the Quantitative-Qualitative Debate: Implications for Mixed-Methods Research (2002) · cites this paper as Greene and Caracelli, 1989
In the citing paper
“The results were measured through a mixed concurrent or triangulated study [19–21].”
From The virtual patient as a learning tool: a mixed quantitative qualitative study (2018) · cites this paper as [19]
In the citing paper
“Likewise, with Greene et al.’s (1989) initiation purpose, the initiation strand follows the unexpected results that it is supposed to explain.”
From How to Construct a Mixed Methods Research Design (2017) · cites this paper as Greene et al.’s (1989)
In the citing paper
“Greene et al. (1989, p. 259) distinguished the following five purposes for mixing in mixed methods research:”
From How to Construct a Mixed Methods Research Design (2017) · cites this paper as Greene et al. (1989)
In the citing paper
“Bryman’s classification breaks down Greene et al.’s (1989) categories into several aspects, and he adds a number of additional aspects, such as the following:”
From How to Construct a Mixed Methods Research Design (2017) · cites this paper as Greene et al. (1989)
Citations
Cites
Convergent and discriminant validation by the multitrait-multimethod matrix.
Campbell, D. T., & Fiske, D. W. (1959) · Psychological Bulletin · doi:10.1037/h0046016 · Cited by 17,116
Qualitative Data Analysis: A Sourcebook of New Methods
Lotto, L. S., Miles, M. B., & Huberman, A. M. (1986) · Educational Evaluation and Policy Analysis · doi:10.3102/01623737008003329 · Cited by 10,357
Mixing Qualitative and Quantitative Methods: Triangulation in Action.
Jick, T. D. (1979) · Administrative Science Quarterly · doi:10.2307/2392366 · Cited by 7,103
The Research Act: A Theoretical Introduction to Sociological Methods
Fox, W. S., & Denzin, N. K. (1979) · Contemporary Sociology A Journal of Reviews · doi:10.2307/2065439 · Cited by 6,295
Qualitative data analysis: A sourcebook of new methods
Dijkstra, L. (1985) · European Journal of Operational Research · doi:10.1016/0377-2217(85)90239-5 · Cited by 2,913
Barrett, M. W. (1995) · Evaluation Practice · doi:10.1016/0886-1633(95)90009-8 · Cited by 961
Rossman, G., & Wilson, B. L. (1985) · Evaluation Review · doi:10.1177/0193841x8500900505 · Cited by 945
Brinberg, D. L., Cook, T. D., & Reichardt, C. S. (1981) · Educational Researcher · doi:10.2307/1174260 · Cited by 911
Mathison, S. (1988) · Educational Researcher · doi:10.3102/0013189x017002013 · Cited by 762
SMITH, J. K., & HESHUSIUS, L. (1986) · Educational Researcher · doi:10.3102/0013189x015001004 · Cited by 675
The Integration of Fieldwork and Survey Methods
Sieber, S. D. (1973) · American Journal of Sociology · doi:10.1086/225467 · Cited by 618
Quantitative Versus Qualitative Research: An Attempt to Clarify the Issue
Smith, J. K. (1983) · Educational Researcher · doi:10.3102/0013189x012003006 · Cited by 608
Dunn, W. N. (1983) · Science Communication · doi:10.1177/0164025983004004007 · Cited by 475
Beyond qualitative versus quantitative methods
Reinhardt, C. S., & Cook, T. (1979) · Cited by 446
Qualitative and quantitative methods: When stories converge
Kidder, L. H., & Fine, M. (1987) · New Directions for Program Evaluation · doi:10.1002/ev.1459 · Cited by 336
Social Science and Social Policy.
Freeman, H. E., Shotland, R. L., & Mark, M. M. (1986) · Contemporary Sociology A Journal of Reviews · doi:10.2307/2070103 · Cited by 303
Post-positivist critical multiplism
Cook, T. D. (1985) · Cited by 253
Greene, J., & McClintock, C. C. (1985) · Evaluation Review · doi:10.1177/0193841x8500900501 · Cited by 171
Self-citationSocial Science and Social Policy
(1976) · Elsevier eBooks · doi:10.1016/b978-0-08-018226-1.50012-7 · Cited by 161
Ethnography in Educational Evaluation
Fetterman, D. M. (1984) · Medical Entomology and Zoology · Cited by 146
Madey, D. L. (1982) · Educational Evaluation and Policy Analysis · doi:10.3102/01623737004002223 · Cited by 93
On The Reconciliation of Qualitative and Quantitative Analyses: A Case Study
Trend, M. (1978) · Human Organization · doi:10.17730/humo.37.4.181457u6563w1187 · Cited by 84
Educational Evaluation: Ethnography in Theory, Practice, and Politics.
Fetterman, D. M., & Pitman, M. A. (1986) · Medical Entomology and Zoology · Cited by 62
Alternative models for the use of multiple methods
Mark, M. M., & Shotland, R. L. (1987) · New Directions for Program Evaluation · doi:10.1002/ev.1461 · Cited by 59
Improving inferences from multiple methods
Shotland, R. L., & Mark, M. M. (1987) · New Directions for Program Evaluation · doi:10.1002/ev.1460 · Cited by 48
The whole is greater: Combining qualitative and quantitative approaches in evaluation studies
Smith, M. L. (1986) · New Directions for Program Evaluation · doi:10.1002/ev.1425 · Cited by 39
Taking uncertainty into account when estimating effects
Reichardt, C. S., & Gollob, H. F. (1987) · New Directions for Program Evaluation · doi:10.1002/ev.1456 · Cited by 36
Research and Managerial Strategies for Integrating Evaluation Research Into Agency Decision Making
Moran, T. K. (1987) · Evaluation Review · doi:10.1177/0193841x8701100503 · Cited by 14
The Development and Trials of a Decision-Making Model
Peters, M. A., Marshall, J. D., & Shaw, R. K. (1986) · Evaluation Review · doi:10.1177/0193841x8601000101 · Cited by 8
Policy Researcher as Sleuth: New Approaches to Integrating Qualitative and Quantitative Methods.
Louis, K. S. (1981) · Cited by 5
Evaluation and Comparison: Social Learning Curriculum and Instrumental Enrichment. Final Report.
Hall, J. (1981) · Cited by 5
Triangulation in the Evaluation of School System Interventions.
Bower, J. C. (1986) · Cited by 1
Flagged first, then by citations
Research works printed but not linked: 19
- Bower JC, 1986, Paper presented at the annual meeting of the American Educational Research Association
- Cook TD, 1979, Qualitative and quantitative methods in evaluation research
- Denzin NK, 1978, The research act: An introduction to sociological methods
- Gray P, 1987, Paper presented at the annual meeting of the American Evaluation Association
- Greene JC, 1987, Proposal for the 1988 annual meeting of the American Education Research Association
- Kidder LH, 1987, Multiple methods in program evaluation: New Directions for Program Evaluation 35, 57
- Knapp MS, 1979, Qualitative and quantitative methods in evaluation research, 118
- Louis KS, 1981, Paper presented at the annual meeting of the American Educational Research Association
- Mark MM, 1988, Panel presented at the annual meeting of the American Evaluation Association
- Mark MM, 1987, Multiple methods in program evaluation: New Directions for Program Evaluation 35, 95
- Martin GO, 1987, Paper presented at the annual meeting of the American Evaluation Association
- 10.2307/1174583 · DOI
- Moody PR, 1982, Pitt Meadows physical education project: Evaluation of an implementation project
- Reichardt CS, 1979, Qualitative and quantitative methods in evaluation research, 7
- Reichardt CS, 1987, Multiple methods in program evaluation: New Directions for Program Evaluation 35, 7
- Shotland RL, 1987, Multiple methods in program evaluation: New Directions for Program Evaluation 35, 77
- Smith ML, 1986, Naturalistic evaluation: New Directions for Program Evaluation 30, 37
- Trend MJ, 1979, Qualitative and quantitative methods in evaluation research, 68
- Webb EJ, 1966, Unobtrusive measures
Cited by
Sorted by citations
Mixed Methods Research: A Research Paradigm Whose Time Has Come
Johnson, R. B., & Onwuegbuzie, A. J. (2004) · Educational Researcher · doi:10.3102/0013189x033007014 · Cited by 11,170
Toward a definition of mixed methods research
Johnson, R. B., Onwuegbuzie, A. J., & Turner, L. A. (2007) · Journal of Mixed Methods Research · doi:10.1177/1558689806298224 · Cited by 6,464
Achieving integration in mixed methods designs-principles and practices
Fetters, M. D., Curry, L. A., & Creswell, J. W. (2013) · Health Services Research · doi:10.1111/1475-6773.12117 · Cited by 7,219
Integrating quantitative and qualitative research: How is it done
Bryman, A. (2006) · Qualitative Research · doi:10.1177/1468794106058877 · Cited by 3,354
Methodological fit in management field research
Edmondson, A. C., & McManus, S. E. (2007) · Academy of Management Review · doi:10.5465/amr.2007.26586086 · Cited by 3,493
Metadata
- Authors
- Greene, J. C., Caracelli, V. J., & Graham, W. F.
- Affiliations
- Cornell University (US)
- Venue
- Educational Evaluation and Policy Analysis
- Published
- 1989-09
- Volume
- 11
- Issue
- 3
- Pages
- 255-274
- ISSN
- 0162-3737 (ISSN-L)
- 1935-1062
- OpenAlex
- W2109685530
- Cited by
- 5,687
- References
- 32
- Citations by year
- 2026: 282 · 2025: 266 · 2024: 273 · 2023: 300 · 2022: 321 · 2021: 377 · 2020: 322 · 2019: 300 · 2018: 266 · 2017: 286 · 2016: 314 · 2015: 336 · 2014: 328 · 2013: 300 · 2012: 279
- FWCI
- 6.74
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Educational Evaluation and Policy Analysis (publishedVersion)
- Topics
- Social Sciences › Public Administration › Evaluation and Performance Assessment
- Social Sciences › Education › Educational Research and Methods
- Social Sciences › Sociology and Political Science › Qualitative Comparative Analysis Research
- Keywords
- conceptual framework, evaluation design, triangulation, complementarity, program evaluation, educational program evaluation, social program evaluation, integrated data analysis, design features
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
