The DOI matches its Crossref record; 18 of the 18 references it cites were found in a registry or an index.
Critical Appraisal of Mixed Methods Studies
Open accessAbstract
In several subdomains of the social, behavioral, health, and human sciences, research questions are increasingly answered through mixed methods studies, combining qualitative and quantitative evidence and research elements. Accordingly, the importance of including those primary mixed methods research articles in systematic reviews grows. It is generally known that the critical appraisal of articles is an essential step in the development of a methodologically sound review. This article provides an overview of the available critical appraisal frameworks developed to evaluate primary mixed methods research articles. In addition, we critically compare and evaluate these frameworks and the quality criteria they include.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Mixed Methods Research: A Research Paradigm Whose Time Has Come
Johnson, R. B., & Onwuegbuzie, A. J. (2004) · Educational Researcher · doi:10.3102/0013189x033007014 · Cited by 11,170 · cited 3×
“…mixed methods research, systematic review, research synthesis In the past decades, the popularity of mixed methods research (MMR) has increased steadily (Creswell, 2003; Greene, 2007; Johnson & Onwuegbuzie, 2004; Onwuegbuzie & Leech, 2005a; Tashakkori & Creswell, 2007; Tashakkori & Teddlie, 2003a).”
MethodSAGE Handbook of Mixed Methods in Social & Behavioral Research
Tashakkori, A., & Teddlie, C. (2010) · doi:10.4135/9781506335193 · Cited by 6,215
Toward a Conceptual Framework for Mixed-Method Evaluation Designs
Greene, J. C., Caracelli, V. J., & Graham, W. F. (1989) · Educational Evaluation and Policy Analysis · doi:10.3102/01623737011003255 · Cited by 5,687
The Handbook of Research Synthesis and Meta-Analysis
(2019) · doi:10.7758/9781610448864 · Cited by 4,553
But is it rigorous? Trustworthiness and authenticity in naturalistic evaluation
Lincoln, Y. S., & Guba, E. G. (1986) · New Directions for Program Evaluation · doi:10.1002/ev.1427 · Cited by 3,668
Systematic Reviews in the Social Sciences: A Practical Guide
Petticrew, M., & Roberts, H. (2006) · doi:10.1002/9780470754887 · Cited by 3,586
Assessing Risk of Bias in Included Studies
Higgins, J. P., & Altman, D. G. (2008) · Cochrane Handbook for Systematic Reviews of Interventions · doi:10.1002/9780470712184.ch8 · Cited by 3,396 · cited 1×
“After all, it is generally known that the inclusion of studies that are methodologically flawed could lead to a substantial bias in the end result of a systematic review (Cooper & Hedges, 1994; Higgins & Altman, 2008).”
BackgroundIntegrating quantitative and qualitative research: How is it done
Bryman, A. (2006) · Qualitative Research · doi:10.1177/1468794106058877 · Cited by 3,354
Realist review - a new method of systematic review designed for complex policy interventions
Pawson, R., Greenhalgh, T., Harvey, G., & Walshe, K. (2005) · Journal of Health Services Research & Policy · doi:10.1258/1355819054308530 · Cited by 3,190 · cited 3×
“The authors who developed frameworks for undertaking systematic reviews applying MM approaches (e.g., Gaber, 2000; Harden & Thomas, 2005, 2010; Heyvaert et al., 2013; Onwuegbuzie et al., 2010; Pawson et al., 2005; Pluye et al., 2009b; Sandelowski et al., 2006; Whittemore & Knafl, 2005) do mention the necessity of crit…”
BackgroundUnderstanding and Validity in Qualitative Research
Maxwell, J. (1992) · Harvard Educational Review · doi:10.17763/haer.62.3.8323320856251826 · Cited by 2,951
Research works printed but not linked: 69
- 10.1111/j.1471-1842.2004.00543.x · DOI
- 10.4135/9781506335193.n4 · DOI · Paper page
- 10.1080/13645570600595280 · DOI · Paper page
- 10.1080/13645570701401644 · DOI · Paper page
- Collins K. M. T., 2006, Learning Disabilities: A Contemporary Journal, 4, 67
- Cooper H. M., 1994, The handbook of research synthesis
- Creswell J. W., 2003, Research design: Qualitative, quantitative, and mixed methods approaches
- Creswell J. W., 2008, Keynote address for the Fourth Annual Mixed Methods Conference
- 10.1177/1558689808330883 · DOI · Paper page
- *Creswell J. W., 2007, Designing and conducting mixed methods research
- Creswell J. W., 2011, Designing and conducting mixed methods research
- 10.3102/0013189X08324091 · DOI
- Dellinger A. B., 2005, Research in the Schools, 12, 41
- 10.1177/1558689807306147 · DOI · Paper page
- *Dybå T., 2007, Proceedings of the First International Symposium on Empirical Software Engineering and Measurement, 225
- 10.1016/j.lisr.2008.04.001 · DOI · Paper page
- 10.1016/S0149-7189(00)00012-4 · DOI
- 10.4324/9781410605337 · doi:10.4324/9781410605337
- *Greene J. C., 2007, Mixed methods in social inquiry
- 10.1177/1558689807309969 · DOI · Paper page
- 10.1177/13563890122209504 · DOI · Paper page
- 10.4135/9781506335193.n5 · DOI · Paper page
- 10.1177/1558689808314622 · DOI
- 10.1111/j.1365-2648.2011.05636.x · DOI · Paper page
- 10.1080/13645570500155078 · DOI · Paper page
- 10.4135/9781506335193.n29 · DOI
- 10.1177/1558689808325771 · DOI
- Heyvaert M., 2011, Research in the Schools, 18, 12
- 10.1007/s11135-011-9538-6 · DOI · Paper page
- Higgins J. P. T., Green S. (Eds.). (2009). Cochrane handbook for systematic reviews of interventions (Version 5.0.2). Retrieved from http://www.cochrane-handbook.org · DOI · Paper page
- 10.1007/s11575-006-0100-z · DOI · Paper page
- 10.1023/B:RIHE.0000027392.94172.d2 · DOI
- 10.1177/1558689806298224 · DOI · Paper page
- 10.1186/1471-2288-4-22 · DOI · Paper page
- Khan K. S., Riet G., Popay J., Nixon J., Kleijnen J. (2001). Undertaking systematic reviews of research on effectiveness (Centre for Reviews and Dissemination Report No. 4, 2nd ed., pp. 1-20). York, England: University of York.
- 10.1177/1558689809345262 · DOI · Paper page
- Letts L., Wilkins S., Law M., Stewart D., Bosch J., Westmorland M. (2007). Critical Review Form–Qualitative Studies (Version 2.0). Retrieved from http://www.srs-mcmaster.ca/Portals/20/pdf/ebp/qualreview_version2.0.pdf
- 10.1016/0147-1767(85)90062-8 · DOI · Paper page
- Major C. H., 2010, An introduction to qualitative research synthesis: Managing the information explosion in social science research
- 10.1177/0002764211433797 · DOI · Paper page
- 10.4135/9781506335193.n8 · DOI
- 10.1037/0003-066X.50.9.741 · DOI · Paper page
- 10.1186/1472-6963-6-28 · DOI · Paper page
- 10.1177/2345678906292462 · DOI · Paper page
- Newman I., 1998, Qualitative-quantitative research methodology: Exploring the interactive continuum
- 10.4135/9781506335193.n21 · DOI · Paper page
- 10.1258/jhsrp.2007.007074 · DOI · Paper page
- Onwuegbuzie A. J., 2010, The Research on Stress and Coping in Education Series: Vol. 5. Toward a broader understanding of stress and coping: Mixed methods approaches, 169
- *Onwuegbuzie A. J., 2006, Research in the Schools, 13, 48
- 10.1007/s11135-009-9289-9 · DOI
- 10.1007/s11135-004-1670-0 · DOI · Paper page
- 10.1080/13645570500402447 · DOI · Paper page
- *Pluye P., 2009, Paper presented at the 37th Annual Meeting of the North American Primary Care Research Group
- 10.1016/j.ijmedinf.2005.05.004 · DOI
- 10.1093/acprof:oso/9780195387216.001.0001 · DOI · Paper page
- 10.1023/B:QUQU.0000043126.25329.85 · DOI
- Sandelowski M., 2006, Research in the Schools, 13, 29
- 10.1177/1077800410364610 · DOI · Paper page
- 10.1177/2345678906293042 · DOI · Paper page
- Tashakkori A., 1998, Mixed methodology: Combining qualitative and quantitative approaches
- Tashakkori A., 2003, Handbook of mixed methods in social & behavioral research
- Tashakkori A., 2003, Handbook of mixed methods in social & behavioral research, 671
- Tashakkori A., 2006, Paper presented at the annual meeting of the American Educational Research Association
- **Tashakkori A., 2008, Advances in mixed methods research: Theories and applications, 101
- **Teddlie C., 2003, Handbook of mixed methods in social & behavioral research, 3
- *Teddlie C., 2009, Foundations of mixed methods research: Integrating quantitative and qualitative approaches in the social and behavioral sciences
- 10.1080/13645570903097950 · DOI
- 10.1177/104973201129119299 · DOI · Paper page
- 10.1111/j.1365-2648.2005.03621.x · DOI · Paper page
Cited by
The Mixed Methods Appraisal Tool (MMAT) version 2018 for information professionals and researchers
Hong, Q. N., Fàbregues, S., Bartlett, G., Boardman, F., Cargo, M., Dagenais, P., Gagnon, M.-P., Griffiths, F., Nicolau, B., O’Cathain, A., et al. (2018) · Education for Information · doi:10.3233/efi-180221 · Cited by 4,737
“Currently, there exist more than 500 critical appraisal tools (i.e., checklists including a list of criteria to judge the quality of a study) (Bai et al., 2012; Crowe & Sheppard, 2011; Heyvaert et al., 2013; Santiago-Delefosse et al., 2016).”
Hong, Q. N., Gonzalez‐Reyes, A., & Pluye, P. (2018) · Journal of Evaluation in Clinical Practice · doi:10.1111/jep.12884 · Cited by 1,107
Improving the content validity of the mixed methods appraisal tool: a modified e-Delphi study
Hong, Q. N., Pluye, P., Fàbregues, S., Bartlett, G., Boardman, F., Cargo, M., Dagenais, P., Gagnon, M.-P., Griffiths, F., Nicolau, B., et al. (2019) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2019.03.008 · Cited by 1,082
“For the MMAT, we retained the criterion #6 (quantitative and qualitative components of the study are effectively integrated) because it was also mentioned in other studies as among the most prevalent criterion for assessing the quality of mixed methods studies [17,29].”
MethodMethodological guidance for the conduct of mixed methods systematic reviews
Stern, C., Lizarondo, L., Carrier, J., Godfrey, C., Rieger, K., Salmond, S., Apóstolo, J., Kirkpatrick, P., & Loveday, H. (2020) · JBI Evidence Synthesis · doi:10.11124/jbisrir-d-19-00169 · Cited by 700
Noyes, J., Booth, A., Flemming, K., Garside, R., Harden, A., Lewin, S., Pantoja, T., Hannes, K., Cargo, M., & Thomas, J. (2018) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2017.06.020 · Cited by 480
Self-citation
Metadata
- Authors
- Heyvaert, M., Hannes, K., Maes, B., & Onghena, P.
- Venue
- Journal of Mixed Methods Research
- Publisher
- SAGE Publications
- Published
- 2013-03-04
- Volume
- 7
- Issue
- 4
- Pages
- 302-327
- ISSN
- 1558-6898 (ISSN-L)
- 1558-6901
- OpenAlex
- W2102121765
- Cited by
- 168
- References
- 18
- FWCI
- 11.83
- Type
- journal article
- Language
- en
- Open access
- green
- Licence
- journals.sagepub.com/page/policies/text-and-data-mining-license (tdm)
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Critical Appraisal of Mixed Methods Studies. https://app.cochat.ai/papers/10.1177/1558689813479449
- The paper
- Heyvaert, M., Hannes, K., Maes, B., & Onghena, P. (2013). Critical Appraisal of Mixed Methods Studies. Journal of Mixed Methods Research. https://doi.org/10.1177/1558689813479449
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.