The Mixed Methods Appraisal Tool (MMAT) version 2018 for information professionals and researchers
Open accessThe DOI matches its Crossref record; 21 of the 21 references it cites were found in a registry or an index; 2 of the 26 sources its claims cite were checked for support.
Abstract
INTRODUCTION: Appraising the quality of studies included in systematic reviews combining qualitative and quantitative evidence is challenging. To address this challenge, a critical appraisal tool was developed: the Mixed Methods Appraisal Tool (MMAT). The aim of this paper is to present the enhance ments made to the MMAT. DEVELOPMENT: The MMAT was initially developed in 2006 based on a literature review on systematic reviews combining qualitative and quantitative evidence. It was subject to pilot and interrater reliability testing. A revised version of the MMAT was developed in 2018 based on the results from usefulness testing, a literature review on critical appraisal tools and a modified e-Delphi study with methodological experts to identify core criteria. TOOL DESCRIPTION: The MMAT assesses the quality of qualitative, quantitative, and mixed methods studies. It focuses on methodological criteria and includes five core quality criteria for each of the following five categories of study designs: (a) qualitative, (b) randomized controlled, (c) nonrandomized, (d) quantitative descriptive, and (e) mixed methods. CONCLUSION: The MMAT is a unique tool that can be used to appraise the quality of different study designs. Also, by limiting to core criteria, the MMAT can provide a more efficient appraisal.
Overview
CoChat Summary
TL;DR
This paper presents version 2018 of the Mixed Methods Appraisal Tool (MMAT), a single critical appraisal tool for assessing the methodological quality of qualitative, quantitative, and mixed methods studies in mixed studies reviews. It describes the tool's 25 criteria across five study design categories, its development history, and guidance for use.
Key findings
The MMAT version 2018 includes a total of 25 criteria and 2 screening questions, covering five categories of study designs: (a) qualitative, (b) randomized controlled trial, (c) nonrandomized, (d) quantitative descriptive, and (e) mixed methods, with five core criteria per category rated as yes, no, or can't tell. The tool was developed from a 2006 first version (15 criteria across four categories), revised in 2011, and further revised into the 2018 version based on usefulness testing with 20 researchers, a literature review on critical appraisal tools, and a modified e-Delphi study with 73 methodological experts. Two interrater reliability studies used 32 and 261 papers respectively and showed the need to clarify criteria, particularly for nonrandomized and qualitative studies. The paper argues the MMAT offers a time-efficient alternative by limiting to core criteria and uniquely includes specific criteria for mixed methods studies, though its criteria are harder to judge because they target methodological rather than reporting quality.
Methods
This is a tool description paper (third in a three-part series on systematic reviews for information professionals) rather than an empirical study; it narrates the development of the MMAT and explains how to apply it. Development drew on a literature review on mixed studies reviews, pilot testing in workshops with four reviewers appraising six studies, two interrater reliability studies (32 and 261 papers), usefulness testing via interviews with 20 researchers, a literature review on critical appraisal tools, and a modified e-Delphi study with 73 methodological experts. Use involves three steps: two optional screening questions, selection of the appropriate design category (or categories), and rating each criterion as yes, no, or can't tell, with detailed presentation of ratings advised over an overall numerical score.
Limitations
The paper states that revision of the MMAT has so far focused on content validity and usefulness, and that further testing on its validity and reliability will be needed; it also notes that modifications may be necessary as evidence develops. It states that MMAT criteria are more difficult to judge than other appraisal tools because they focus on methodological quality rather than reporting quality. Evident from the text: the tool's development evidence base is described only in summary form, with reliability and validity results reported by reference to prior studies rather than presented here.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
2 Valid · 0 Not valid · 18 Not checked · 11 Own results and interpretations
Every source the paper cites for this claim was checked and supports it.
In the paper
“To further its development, usefulness testing was performed by interviewing 20 researchers who have used the MMAT in a systematic review or have contacted the developer of the tool for questions or permission to use the tool (18).”
What the sources say
- Supportedcites [18] 18. Hong QN, Gonzalez-Reyes A, Pluye P. Improving the usefulness of a tool for appraising the quality of qualitative, quantitative and mixed methods studies, the Mixed Methods Appraisal Tool (MMAT). J Eval Clin Pract. 2018;24(3):459-67. source
The Abstract reports semi-structured interviews with 20 participants to explore the MMAT’s usefulness. The Study Participants section says researchers were recruited through systematic reviews that used the MMAT and a list of people who had contacted its developer for permission or guidance. (LLM, conf=0.94)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“The results of this study were helpful to identify changes to be made in the MMAT.”
What the sources say
- Supportedcites [18] 18. Hong QN, Gonzalez-Reyes A, Pluye P. Improving the usefulness of a tool for appraising the quality of qualitative, quantitative and mixed methods studies, the Mixed Methods Appraisal Tool (MMAT). J Eval Clin Pract. 2018;24(3):459-67. source
The Discussion says the study explored MMAT users’ experiences to help revise the tool and identified themes suggesting areas for improvement. The Recommendations for Improving the MMAT then proposes specific changes on the basis of those results. (LLM, conf=0.94)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
In the paper
“Moreover, it includes specific criteria for mixed methods studies, which is not often found in other tools (21).”
What the sources say
- Not matchedcites [21] 21. Pluye P. Critical appraisal tools for assessing the methodological quality of qualitative, quantitative and mixed methods studies included in systematic mixed studies reviews. J Eval Clin Pract. 2013;19(4):722.
We searched by its title and found no matching paper.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Not matched
In the paper
“To address this challenge, a critical appraisal tool for assessing the quality of quantitative, qualitative and mixed methods studies was developed: the Mixed Methods Appraisal Tool (MMAT) (7).”
What the sources say
- Not checked — no full text of the sourcecites [7] 7. Pluye P, Gagnon MP, Griffiths F, Johnson-Lafleur J. A scoring system for appraising mixed methods research, and concomitantly appraising qualitative, quantitative and mixed methods primary studies in mixed studies reviews. Int J Nurs Stud. 2009;46(4):529-46. source
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“Moreover, it includes specific criteria for mixed methods studies, which is not often found in other tools (21).”
What the sources say
- Not matchedcites [21] 21. Pluye P. Critical appraisal tools for assessing the methodological quality of qualitative, quantitative and mixed methods studies included in systematic mixed studies reviews. J Eval Clin Pract. 2013;19(4):722.
We searched by its title and found no matching paper.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Not matched
5 more claims
Create a free account to see all 31 claims and to verify the 1 reference still open.
- It may state two separate claims as one. 7
- Probably not a substantial claim. 16
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Quality was assessed using two validated tools by study design. Quantitative studies were appraised using the Mixed Methods Appraisal Tool Version 2018, which evaluates five methodological criteria on a binary scale (Hong et al., 2018). Qualitative studies were additionally appraised using the Critical Appraisal Skills Programme Qualitative Checklist, which evaluates 10 criteria (CASP UK, 2018).”
From Cultural stigma and caregiver burden among family caregivers of people with nervous system disorders in the Middle East and North Africa: A mixed-methods systematic review (2026) · cites this paper as Hong et al., 2018
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“The quality of the included studies was assessed using the 2018 version of the Mixed Methods Appraisal Tool (MMAT).15”
From Psychosocial interventions for siblings of children with cancer: A mixed methods systematic review (2021) · cites this paper as [15]
The cited source says what the claim attributes to it.
In the citing paper
“To further limit risks of bias, a formal risk of bias assessment tool (ROB-tool), the Mixed Methods Appraisal Tool [52], was used to evaluate the overall study quality.”
From Mediators of outcome in adolescent psychotherapy and their implications for theories and mechanisms of change: a systematic review (2024) · cites this paper as [52]
The cited source says what the claim attributes to it.
In the citing paper
“LM, with the support of a research assistant, assessed the methodological quality of empirical studies (qualitative, quantitative and mixed methods) using the Mixed Methods Appraisal Tool (MMAT) [45].”
From Transnationalism and care of migrant families during pregnancy, postpartum and early-childhood: an integrative review (2020) · cites this paper as [45]
The cited source says what the claim attributes to it.
In the citing paper
“The quality of the included studies was independently assessed by two authors using the Mixed Methods Appraisal Tool (MMAT) (Hong et al., 2018).Disagreements were resolved through discussion.”
From Gamification on Mathematics Engagement and Motivation in Secondary School and Higher Education: A Systematic Review and Meta-Analysis (2026) · cites this paper as Hong et al., 2018
29 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ 4. Conclusion · p. 5
“## 4. Conclusion”
§ 4. How to use the MMAT? · p. 4
“## 4. How to use the MMAT?”
Citations
Cites
Pace, R., Pluye, P., Bartlett, G., Macaulay, A. C., Salsberg, J., Jagosh, J., & Seller, R. (2012) · International Journal of Nursing Studies · doi:10.1016/j.ijnurstu.2011.07.002 · Cited by 1,331 · cited 2×
“Then, two interrater reliability studies on the MMAT were conducted using respectively 32 and 261 papers (Pace et al., 2012; Souto et al., 2015).”
Self-citationPluye, P., Gagnon, M.-P., Griffiths, F., & Johnson-Lafleur, J. (2009) · International Journal of Nursing Studies · doi:10.1016/j.ijnurstu.2009.01.009 · Cited by 1,119 · cited 3×
“This led to develop a first version of the MMAT that is in line with a social constructionist worldview (Pluye et al., 2009).”
InfluentialSelf-citationHong, Q. N., Gonzalez‐Reyes, A., & Pluye, P. (2018) · Journal of Evaluation in Clinical Practice · doi:10.1111/jep.12884 · Cited by 1,107 · cited 3×
“To further its development, usefulness testing was performed by interviewing 20 researchers who have used the MMAT in a systematic review or had contacted the developer of the tool for questions or permission to use the tool (Hong et al., 2018a).”
Self-citationSystems to rate the strength of scientific evidence.
West, S. L., King, V., Carey, T. S., Lohr, K. N., Mckoy, N. A., Sutton, S. F., & Lux, L. J. (2002) · PubMed Central · Cited by 923
Zaza, S., Wright-De Agüero, L. K., Briss, P. A., Truman, B. I., Hopkins, D. P., Hennessy, M. H., Sosin, D. M., Anderson, L., Carande-Kulis, V. G., Teutsch, S. M., et al. (2000) · American Journal of Preventive Medicine · doi:10.1016/s0749-3797(99)00122-1 · Cited by 464 · cited 1×
“The algorithm was developed based on several existing algorithms of quantitative study designs (Hartling et al., 2010; Hartling et al., 2011; National Institute for Health Care Excellence, 2012; Scottish Intercollegiate Guidelines Network; Seo et al., 2016; West et al., 2002; Zaza et al., 2000).”
Souto, R. Q., Khanassov, V., Hong, Q. N., Bush, P. L., Vedel, I., & Pluye, P. (2015) · International Journal of Nursing Studies · doi:10.1016/j.ijnurstu.2014.08.010 · Cited by 375 · cited 1×
“Then, two interrater reliability studies on the MMAT were conducted using respectively 32 and 261 papers (Pace et al., 2012; Souto et al., 2015).”
Self-citationA review of critical appraisal tools show they lack rigor: Alternative tool structure is proposed
Crowe, M., & Sheppard, L. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2010.02.008 · Cited by 261 · cited 2×
“Currently, there exist more than 500 critical appraisal tools (i.e., checklists including a list of criteria to judge the quality of a study) (Bai et al., 2012; Crowe & Sheppard, 2011; Heyvaert et al., 2013; Santiago-Delefosse et al., 2016).”
Carroll, C., Booth, A., & Lloyd-Jones, M. (2012) · Qualitative Health Research · doi:10.1177/1049732312452937 · Cited by 248 · cited 1×
“Methodological quality criteria are more difficult to interpret because the reviewers need to judge whether the reported results of a study are trustworthy (Carroll et al., 2012; Hong & Pluye, 2018a).”
Adjustment of meta-analyses on the basis of quality scores should be abandoned
Herbison, P., Hay-Smith, J., & Gillespie, W. J. (2006) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2006.03.008 · Cited by 207 · cited 1×
“Currently in the literature, it is discouraged to calculate an overall score from the ratings of each criterion (Herbison et al., 2006; Higgins & Green, 2008; Viswanathan et al., 2012).”
Critical Appraisal of Mixed Methods Studies
Heyvaert, M., Hannes, K., Maes, B., & Onghena, P. (2013) · Journal of Mixed Methods Research · doi:10.1177/1558689813479449 · Cited by 168 · cited 1×
“Currently, there exist more than 500 critical appraisal tools (i.e., checklists including a list of criteria to judge the quality of a study) (Bai et al., 2012; Crowe & Sheppard, 2011; Heyvaert et al., 2013; Santiago-Delefosse et al., 2016).”
Cited by
Items 3,701–3,800 of 4,578
Clima ético en el entorno de trabajo en la atención primaria de la salud: un estudio mixto
Pereira, L. E. M., Brehmer, L. C. D. F., dos Santos, J. L. G., et al. (2026) · Revista Latino-Americana de Enfermagem · doi:10.1590/1518-8345.7499.4755
Gerosideris, N., Daskalou, S. D., Stratou, E., et al. (2026) · OTJR Occupational Therapy Journal of Research · doi:10.1177/15394492251409640
Clima ético no ambiente de trabalho na Atenção Primária à Saúde: um estudo misto
Pereira, L. E. M., Brehmer, L. C. D. F., dos Santos, J. L. G., et al. (2026) · Revista Latino-Americana de Enfermagem · doi:10.1590/1518-8345.7499.4757
Ethical climate in the primary health care workplace: a mixed-method study
Pereira, L. E. M., Brehmer, L. C. D. F., dos Santos, J. L. G., et al. (2026) · Revista Latino-Americana de Enfermagem · doi:10.1590/1518-8345.7499.4756
Toledo, I. D. F. B., Caldas, C. P., & Camacho, A. C. L. F. (2026) · Revista Contemporânea · doi:10.56083/rcv6n1-067
Braën-Boucher, C., & Roberge, M.-C. (2026) · doi:10.64490/rouu6008
Effects of dental implants on dietary and nutritional outcomes in older adults: a systematic review
Induijaa, S., Senevirathna, K., Batuwanthudawa, M., Kodikarage, S. C. H., Bandara, P., Bandara, D. L., Kanmodi, K. K., Jayasinghe, R. D., & Jayasinghe, R. M. (2026) · BMC Oral Health · doi:10.1186/s12903-026-07708-w · Cited by 2
“These participants tended to be slightly younger within older adult category, with mean ages ranging from 62.6 to 81.1 years, and study-specific age ranges extending from 55 to 103 years [10, 15, 25, 30].”
InfluentialHagen, M. H., Jaccheri, L., & Hartvigsen, G. (2026) · doi:10.2196/preprints.89825
The Power of Literature Reviews
Willgerodt, M. A. (2026) · The Journal of School Nursing · doi:10.1177/10598405251409622
Prasetia, A. R., Perdananti, P., Waspada, I., & Sari, M. (2026) · Moneta Journal of Economics and Finance · doi:10.61978/moneta.v4i1.1249
“Methodological quality and risk of bias were assessed after full-text inclusion using a ten-item 0/1 rubric operationalised for cross-design comparability (Hong et al., 2018).”
Byrne, A., Hall, A., Cutmore, E., & Mulvogue, J. (2026) · Nursing Open · doi:10.1002/nop2.70395 · Cited by 1
“Given that studies were not included or excluded based on methodology or methods, the MMAT was useful in determining the quality of each included study, allowing a rigorous assessment of the research question, study, and findings.”
Patient perspectives on telehealth access among people experiencing homelessness: a rapid review
Doherty, C., Pepper, H., Jessup, R. L., & Hutton, J. (2026) · International Journal for Equity in Health · doi:10.1186/s12939-025-02749-z · Cited by 1
“The Mixed Methods Appraisal Tool (MMAT) [27] was used to assess the quality of each paper.”
Renaldi, F. S., & Pangeran, K. (2026) · Journal of Pharmaceutical And Sciences · doi:10.36490/journal-jps.com.v9i1.1331
Nguyen, T. T. L., Muller, A., & Geller, G. (2026) · Sexual Health · doi:10.1071/sh25051
Jayasinghe, M. S., Warnasekara, J. N., Gunarathna, S. P., et al. (2026) · BMJ Open · doi:10.1136/bmjopen-2025-106730
Anand, R., Nag, D. S., Gope, R. L., Sahoo, M. K., Bhushan, P., Pal, B. D., Patel, R., Srirala, S., Bharadwaj, M. K., & Ansari, M. A. (2026) · World Journal of Psychiatry · doi:10.5498/wjp.v16.i2.112462 · Cited by 2
Yan, Y., Zheng, Q., Miao, X., & Wei, Y. (2026) · Humanities and Social Sciences Communications · doi:10.1057/s41599-026-06578-5 · Cited by 1
“This validated tool facilitates evaluation of fi ve study designs: qualitative research, randomized controlled trials, non-randomized studies, quantitative descriptive studies, and mixed-methods research(Hong et al., 2018).”
Barriers to Childhood Immunisation in Nigeria: A Scoping Review of Recent Empirical Studies
Hamina, D., Sharoni, S. K. A., Kever, R. T., & Joseph-Shehu, E. M. (2025) · Malaysian Journal of Medicine and Health Sciences · doi:10.47836/mjmhs.21.2.28
Reuben, W., Nyondo, G. G., Sangeda, R. Z., & Ngoma, S. J. (2026) · SSM - Health Systems · doi:10.1016/j.ssmhs.2026.100183 · Cited by 1
Drew, K. E., Timba-Emmanuel, T., & Lindsay, I. (2026) · Nurse Education in Practice · doi:10.1016/j.nepr.2026.104745
CENÁRIO DA REALIDADE AUMENTADA NO ENSINO DE ELETROMAGNETISMO: UMA REVISÃO DE 2020 A 2024
da Silva, D. G., & Rivera, J. A. (2025) · REAMEC - Rede Amazônica de Educação em Ciências e Matemática · doi:10.26571/reamec.v13.19724
Cárdenas, C. A. T., Palma, A. I. C., García, M. Y. V., Mora, J. D. R. P., & Vélez, C. M. V. (2026) · Revista Multidisciplinar Epistemología de las Ciencias · doi:10.71112/dw6xw952
Lima, M. I. C. M., Melo, G., & Pinto, S. M. O. (2026) · Systematic Reviews · doi:10.1186/s13643-025-03012-w
“For mixed methods studies, we may alternatively use the Mixed Methods Appraisal Tool [46], depending on which is most appropriate.”
Waratworawan, W., Witzel, T. C., Lim, S. H., & Guadamuz, T. E. (2026) · BMC Medical Ethics · doi:10.1186/s12910-026-01378-7
“Based on the MMAT score of 0–5, studies were categorized as follows: those with a score of 0–2 were rated as low quality, studies scoring 3–4 were classified as moderate quality, and studies with a score of 5 were considered to be of high quality [49–51].”
Selak, M. B., Merkaš, M., & Žulec, A. (2025) · Dijete i obitelj u suvremenom društvu · doi:10.25234/dosd/38182 · Cited by 1
“The study quality is assessed with the study quality instrument, the Mixed Methods Appraisal Tool (MMAT; Hong et al. , 2018).”
González, N. V. Y., Moreno, N., Botero, D., & Chaparro, S. A. (2025) · Congreso Internacional de Innovación y Tendencias en Ingeniería (CONIITI) · doi:10.1109/coniiti68560.2025.11366162
Edwards, D. J., Clark, E., Csontos, J. K., Dobbins, M., Gillen, E., Hounsome, J. C., Neil‐Sztramko, S., Lewis, R., Mann, M. K., & Prue, G. (2026) · Research Square · doi:10.21203/rs.3.rs-8724550/v1
Petruzzi, L. J., Mandalapu, A., Mang, B., Quan, E., Vetter, I., Collier, J., Garay, R., Yokananth, R., Valdez, C. R., Cook, R., et al. (2026) · International Journal for Equity in Health · doi:10.1186/s12939-026-02770-w
“Study design was categorized based on the Mixed Methods Appraisal Tool [44] and included non-randomized quantitative studies ( n = 35), descriptive quantitative studies, ( n = 34), RCTs ( n = 27), qualitative studies ( n = 27), mixed methods studies ( n = 13).”
Alexander, E. D., Sarmiento, I., Chung, V.-H.-A., Yacovelli, A., & Andersson, N. (2026) · BMJ Open · doi:10.1136/bmjopen-2025-107379
“36 It allowed assessing the quality of included studies based on their type (qualitative/quantitative-randomised, quantitative- non-randomised, quantitative descriptive and mixed methods).”
Ahmed, B. H., Voss, J. G., Griggs, S. A., et al. (2026) · Chronic Illness · doi:10.1177/17423953261417151
“13 The rationale for using the MMAT is that it is helpful to equally evaluated similar items across the different study designs.”
Psychological recovery of nurses after emergencies and disasters: a systematic review protocol
Shahrestanaki, Y. A., Hosseini, F., Kashiha, E., Norouzi, S., Jirandehi, F. M., & Beyramijam, M. (2026) · BMJ Open · doi:10.1136/bmjopen-2025-105202
“Therefore, the methodological quality and risk of bias of included studies will be assessed using the MMAT, 24 which is appropriate for evaluating qualitative, quantitative and mixed-methods studies.”
Cedillo-Hernández, A., Velázquez-García, L., Longar-Blanco, M. D. P., Cedillo-Hernández, M., & Lopez-Gonzalez, M. G. C. (2026) · Education Sciences · doi:10.3390/educsci16020256
Alshammari, B. K. H., Alharbi, K. R. F., Alanazi, M. M., et al. (2024) · International Journal of Computational and Experimental Science and Engineering · doi:10.22399/ijcesen.4855
“In an inpatient setting, this might involve a daily nursing oral assessment for high-risk patients (e.g., oncology, critical care) with automatic consultation to a hospital-based dentist or dental hygienist if scores exceed a certain threshold [14].”
Kalu, M. E., Okeke, C. O., Nwachukwu, E. C., Okoh, A., Akinrolie, O., Ezulike, C., Adandom, H., Onyeso, O. K., Egbumike, J., Olatunji, F. D., et al. (2022) · Journal of Global Health Economics and Policy · doi:10.7189/001c.36188
Ros, N. M., & Noor, M. S. Z. (2026) · Multidisciplinary Reviews · doi:10.31893/multirev.2026368 · Cited by 1
“This evaluation protocol was chosen in line with the opinion of Hong et al. (2018) that MMAT is able to systematically evaluate reviews of mixed studies, including five types of studies: qualitative studies, randomized controlled trials, nonrandomized studies, descriptive quantitative studies, and…”
Rural Women’s Experiences and Motivations in Charcoal Production and Trade in Ghana
Beyuo, A., Suleman, S., & Bebelleh, F. D. (2026) · Springer eBooks · doi:10.1007/978-3-030-68127-2_882-1
May, R., Rogers, M., & ANDERSON, J. (2026) · Mental Health Practice · doi:10.7748/mhp.2026.e1768
Determinants of firms’ acceptability of carbon taxation: a systematic literature review
Mwang’onda, E. S., Steenkamp, L., & Terblanche‐Smit, M. (2026) · Climate Policy · doi:10.1080/14693062.2026.2627746
Mobile Applications for Enhancing Emotional Wellness: A Systematic Review
Castle, M., Ohrt, J. H., Sookwah, R. D., LoDato, K. K., Jerideau, S., Horton, K., Craven, L. C., Combs, E. M., & Tolan, L. (2026) · Journal of Mental Health Counseling · doi:10.17744/mehc.48.1.01 · Cited by 1
Aminu, K., Jayasinghe, Y. A., Adegbile, O. E., Olorunlana, A., Nwebo, R. A., Nnannah, P. C., Salami, A. A., Adisa, A. O., Eze, U. A., & Kanmodi, K. K. (2026) · Health Science Reports · doi:10.1002/hsr2.71750 · Cited by 1
Alhabshi, E., Ritchie, D., & Hemingway, P. (2026) · Research Square · doi:10.21203/rs.3.rs-7397117/v1
Haider, S. A., Prabha, S., Gomez-Cabello, C. A., Genovese, A., Collaço, B. G., Wood, N. G., Lifson, M. A., Tao, C., Bagaria, S. P., & Forte, A. J. (2026) · doi:10.2196/preprints.93344
Participar no siempre es incluir: poder, reconocimiento e implementación de metodologías activas
Alvarado, G. A. Z. (2026) · Pedagogical Constellation · doi:10.69821/constellations.v5i1.151
Mental health and well-being in youth football players: a systematic review
Demirsöz, G., Scaramuzza, C., Raimo, M., Yiapanas, G., Hamdi, F., Berchicci, M., & Bertollo, M. (2026) · Sport Sciences for Health · doi:10.1007/s11332-026-01677-x · Cited by 2
“To assess the potential risk of bias, the methodological quality of the included studies was evaluated using the Mixed Methods Appraisal Tool (MMAT) [22].”
McEvoy, D., Sweeney, C., Ryan, L., Wilson, C. M., Healy, C., Mongan, D., Sheaf, G., Higgins, A., Gallant, A. J., Doyle, L., et al. (2026) · Journal of Psychiatric Research · doi:10.1016/j.jpsychires.2026.02.027
Peacock, R., Bedding, C., Pacheco, B., Hamzeh, H., Piccinin, C., & Gilbert, A. (2026) · JNCI Journal of the National Cancer Institute · doi:10.1093/jnci/djag039
Cumulative war-related trauma in the MENA region: a systematic review
Abi‐Habib, R., Chahine, M., El‐Khoury, J., Choucair, N., Badr, E., abele, M., Moujabber, J. E., & Jamaleddin, J. M. (2026) · International Review of Psychiatry · doi:10.1080/09540261.2026.2629463 · Cited by 1
“As for the quality assessment of the mixed-methods study, The Mixed Methods Appraisal Tool was utilized to appraise the methodological quality of the study combining both quantitative and qualitative evidence (Hong et al., 2018).”
Devadoss, F., Priyadarshini, R. G., & Jena, L. K. (2026) · Management and Labour Studies · doi:10.1177/0258042x261421683
Khunaifi, A., Handoyo, K. M., Wijananto, F., et al. (2025) · doi:10.1109/icaitech66481.2025.11387644
“Quality assessment frameworks employed the Modified Mixed Methods Appraisal Tool (MMAT) [9] criteria with inter-rater reliability measures attaining Cohen's κ = 0.89, showing excellent agreement between reviewers.”
InfluentialBassey, A. E., Kanu, P. A., Imo, U. F., Akpan, U. U., Adebisi, Y. A., & Nwosu, C. D. (2026) · BMC Medical Education · doi:10.1186/s12909-026-08681-2 · Cited by 2
“Studies were appraised for quality using the Mixed Meth-ods Appraisal Tool (MMAT) Version 2018 [21].”
A systematic review of interventions for developing athletes as leaders
Butalia, R., Rupp, K.-S., Tuand, K., Boen, F., Steffens, N. K., Bongaerts, L., Loughead, T. M., Schepers, K., Hendrickx, M.-L., & Fransen, K. (2026) · International Journal of Sport and Exercise Psychology · doi:10.1080/1612197x.2025.2600983 · Cited by 3
Adlung, C., Niehot, C. D., & van Capelle, C. I. (2026) · BMJ Open · doi:10.1136/bmjopen-2025-112005
Cakar-Ozcelik, M., Usta, E., & İnözü, M. (2026) · Current Psychology · doi:10.1007/s12144-026-09120-7 · Cited by 1
“Two reviewers independently assessed the studies using the MMAT, providing responses of “yes,” “no,” or “can’t tell”.”
InfluentialEnvironmental Justice and Equity in Canadian Nursing for Planetary Health: An Integrative Review
Jones, A. T., Josewski, V., Dewell, S., & Astle, B. (2026) · Canadian Journal of Nursing Research · doi:10.1177/08445621251413667
“This review employed multiple appraisal tools to evaluate diverse sources: the Mixed Methods Appraisal Tool (Hong et al., 2018) for empirical studies, the Joanna Briggs Institute (JBI) Critical Appraisal Checklist for Text and Opinion Papers (JBI, 2019) for review and discussion papers, SANRA for…”
Bonnamy, J., Dart, J., Palermo, C. E., Molloy, R., Small, K., & Connell, C. J. (2026) · BMJ Quality & Safety · doi:10.1136/bmjqs-2025-019529 · Cited by 1
SMEs and Supply Chains: A Systematic Review of CSR Adoption and Scalability ModelsPreprint
Ndemena, P. (2026) · Research Square · doi:10.21203/rs.3.rs-8916003/v1
Talbot, R., Harris, C., Cable, E., Whiten, B., Warren, J. D., Varley, R. A., & Volkmer, A. (2026) · Dementia · doi:10.1177/14713012261425555 · Cited by 3
Rane, E., Bloom, J., Chan, C., & Harris, P. (2026) · Pediatric Surgery International · doi:10.1007/s00383-026-06307-8
“However, quality appraisal alone did not warrant exclusion of articles, as overall score calculations are discouraged in context of differing approaches contributing subjectively to the overall field of interest [ 17].”
Warning response behavior model: conceptual clarification and operationalization
Xu, H. H., Lu, Y., Chen, Y., & Fan, Y. (2026) · Natural Hazards · doi:10.1007/s11069-026-08003-1 · Cited by 1
“Simultaneously, we assessed the quality of the candidate literature using the Mixed Methods Appraisal Tool (Hong et al. 2018) and further screened the candidate literature based on this evaluation.”
Lihemo, G., Blunt, M., Dadari, I., Underwood, T., Toasa, A. E. O., Velias, A., Hopkins, K. L., Thomson, A., Kanwagi, R., Gillespie, A. M., et al. (2026) · Frontiers in Public Health · doi:10.3389/fpubh.2026.1764389 · Cited by 1
Utilization of Campus Recreation Programs Among University Students: A Systematic Review
Sim, J., Konnerth, A. N., Yun, S., & Kirk, T. N. (2026) · Recreational Sports Journal · doi:10.1177/15588661261424188 · Cited by 3
“Each article was appraised for methodological quality using the Mixed Methods Appraisal Tool (MMAT; Hong et al., 2018), which accommodates qualitative, quantitative, and mixed-methods designs.”
Figueira, F. J. A., Tavares, M. D. S. F., Andreucci, C. B., Souto, B. G. A., & Osório, N. S. (2026) · Systematic Reviews · doi:10.1186/s13643-026-03091-3
“Consistent with a convergent mixed-methods design [29, 30], quantitative and qualitative evidence will undergo parallel analysis followed by integration.”
van Vliet, M. H. M., Penfornis, K. M., Gebhardt, W. A., von Haeseler, E. F., Chavannes, N. H., Versluis, A., & Meijer, E. (2026) · Systematic Reviews · doi:10.1186/s13643-026-03103-2 · Cited by 3
“However, we selected the Mixed Methods Appraisal Tool (MMAT; version 2018) [48] as it combines assessments for different study designs into a single standardized tool.”
Thomson, G., Ceballos-Rasgado, M., Harris, C., & Schroeder, D. (2026) · Wellcome Open Research · doi:10.12688/wellcomeopenres.24827.3 · Cited by 1
Environmental exposures and cognitive-psychosocial development of children in Asia: A scoping review
Nagarajan, S., P., F. H. K., Holla, B., Sharma, E., Goyal, A. K., Christy, J., Bhadra, S., & Damodharan, D. (2026) · Geopsychiatry · doi:10.1016/j.geopsy.2026.100058
O’Callaghan, E., & McKay, E. A. (2026) · Irish Journal of Occupational Therapy · doi:10.1108/ijot-10-2025-0041
Intergenerational Co-parenting and Children’s Screen Use: A Scoping Review
Xie, Z., Chu, J. T. W., Goel, A., & Thabrew, H. (2026) · Clinical Child and Family Psychology Review · doi:10.1007/s10567-026-00560-y
“The quality of the studies included in this scoping review was assessed by two authors (Z.X. and A.G.) using the Mixed-Methods Appraisal Tool (MMAT; Hong et al., 2018).”
Dhir, A. M., Singh, A., Clipman, S. J., & Farley, J. E. (2026) · AIDS and Behavior · doi:10.1007/s10461-026-05092-8 · Cited by 1
“For mixed methods studies where data were extracted from both quantitative and quali - tative components, we used the Mixed Methods Appraisal Tool (MMAT) to assess quality [15].”
The application of process mining for clinical pathways: a systematic literature review protocol
Murazzano, L., Landa, P., Bergeron, F., Côté, A., Gartner, J., & Raki, M. H. (2026) · Systematic Reviews · doi:10.1186/s13643-026-03144-7
“To ensure consistency in the application of the MMAT, a calibration exercise will be conducted using a small subset of studies prior to the formal quality assessment [29].”
Andersen, F. B., Pallesen, S., Egaas, I. T., Vedaa, Ø., Nielsen, M. B., Sunde, E., Holmelid, Ø., Kysnes, B. B., Bjorvatn, B., Waage, S., et al. (2026) · Systematic Reviews · doi:10.1186/s13643-026-03146-5
“As most of the studies are expected to be non-randomized cohort studies, cross-sectional, and case-control designs, the Mixed Methods Appraisal Tool (MMAT) for systematic mixed study reviews will be used [31].”
Schneider, K. M., LoDato, K. K., Hodge, V., Perez, L. M., Skeen, A. I., & Ohrt, J. H. (2026) · The Journal for Specialists in Group Work · doi:10.1080/01933922.2025.2599763 · Cited by 1
“To assess the methodological quality of the included studies, we used the Mixed Methods Appraisal Tool (MMAT; Hong et al., 2018).”
InfluentialFactors influencing the retention of Generation Z: A systematic review
Wei, J., & Yazid, Z. (2026) · Multidisciplinary Reviews · doi:10.31893/multirev.2026444 · Cited by 4
“To save the authors from having to select different critical appraisal tools for every kind of study design, a task that can be time-consuming, this unique tool was developed to help with the complicated process of critical appraisal in reviews of systematic mixed studies (Hong et al., 2018).”
Delgado, J., Toledo‐Chávarri, A., Torres‐Castaño, A., Ramos‐García, V., Sacks, A. A., Rivero‐Santana, A., Herrera‐Ramos, E., & Perestelo‐Pérez, L. (2026) · doi:10.2196/preprints.94501
Oral health of incarcerated young people: a scoping review
Mostafa, M., Derbi, H., Kinner, S. A., Laslett, A., Stoove, M., & Crombie, F. A. (2026) · European Archives of Paediatric Dentistry · doi:10.1007/s40368-026-01192-0 · Cited by 1
“Study methodological quality was assessed using the Mixed Method Appraisal Tool (MMAT) (Hong et al. 2018) and classified as low, moderate, or high quality according to the checklist criteria (see supplementary material A).”
Autism in Viet Nam: A systematic scoping review
Boyle, L., Perepa, P., Thalia, K., & Crane, L. (2026) · Autism · doi:10.1177/13623613261425838
“Quality appraisal assessment was completed by the first author, using the Mixed Methods Appraisal Tool (MMAT; Hong et al., 2018).”
Interventions to improve mental well-being and sleep in paramedics: A scoping review
Wanstall, S. E., Brown, B. W. J., Crowther, M., Dunbar, C., Adams, R. J., Naweed, A., & Reynolds, A. C. (2026) · PLoS ONE · doi:10.1371/journal.pone.0344377 · Cited by 1
“[60] Each study was assessed against the MMAT criteria relevant to its design type (qualitative, quantitative, or mixed-methods), with judgments made regarding methodological rigor, including clarity of research questions, appropriateness of data collection and analysis, and risk of bias.”
Hospitalists’ collaborative practices with different stakeholders: A systematic literature review
Mamun, T. I., Alam, L., & Novak, L. L. (2026) · Journal of Patient Safety and Risk Management · doi:10.1177/25160435261429391
Graham, C., Murphy, G., Lee, S. C. T., Clarke, M., & Gaynor, K. (2026) · Early Intervention in Psychiatry · doi:10.1111/eip.70166 · Cited by 2
“All eight studies were reviewed against the MMAT quality appraisal tool (Hong et al. 2018).”
InfluentialTele-Palliative Care in Rural Areas, Implementation and Patient Experiences: A Systematic Review
Pitis, A., Nikoloudi, M., & Mystakidou, K. (2026) · American Journal of Hospice and Palliative Medicine® · doi:10.1177/10499091261434574
Anugrah, S. D., Tanggalong, Y. M., & Anton, A. (2026) · Proceeding International Student Conference · doi:10.24176/insef.v2i1.1260
Pereira, E., Serrada, I., & Guerin, M. (2026) · Physical Therapy Reviews · doi:10.1080/10833196.2026.2642540
“Peer-reviewed studies included in the review, were appraised independently by two reviewers using the Mixed Methods Appraisal Tool (MMAT) [22].”
InfluentialRol del coordinador y acompañamiento pedagógico mediado por tecnología: una revisión narrativa
Camilo, A. S. C., & Estrada, A. F. (2026) · Pedagogical Constellation · doi:10.69821/constellations.v5i1.152
Treating Dissociation in PTSD: A Meta-Analysis of Psychological Intervention Studies
Akoral, M., Swan, A. H., Rizeq, J., & Gumley, A. I. (2026) · Journal of Trauma & Dissociation · doi:10.1080/15299732.2026.2641635 · Cited by 2
“The mixed methods appraisal tool (MMAT, Hong et al., 2018) was used for Harned et al. (2021, see Table 2. for domains).”
Tele-Dance Interventions for Health Outcomes: A Scoping Review
Shim, M., Park, S. G., Smith, D., Uhler, E., & Lacson, C. J. (2026) · Telemedicine Journal and e-Health · doi:10.1177/15305627261427355
“The methodological quality of the included studies was assessed using criteria adapted from the Mixed Methods Appraisal Tool (MMAT), 26 which is designed for use in systematic reviews incorporating qualitative (QUAL), quantitative (QUAN), and mixed-methods (MM) studies.”
Damsté, C., Stoorvogel, H. H., Oerlemans, A. J. M., Knippenberg, M. L., van Gurp, J. L. P., ten Oever, J., & Hulscher, M. (2026) · Journal of Antimicrobial Chemotherapy · doi:10.1093/jac/dkag059 · Cited by 1
Lyon, C., Williams, C., Buswell, S., et al. (2026) · Journal of Huntington s Disease · doi:10.1177/18796397261428341
Khodashenas, Z., & Pourkalhor, H. (2026) · Journal of Skin and Sexually Transmitted Diseases · doi:10.25259/jsstd_148_2025
Liyanage, T. C., & Mallawatantri, S. (2025) · International Conference on Computer and Applications (ICCA) · doi:10.1109/icca66035.2025.11431001
“The methodological quality of the included studies was appraised using the Mixed Methods Appraisal Tool (MMAT, 2018) [10].”
Mussa, I., & Dagada, R. (2026) · Digital Health · doi:10.1177/20552076261427507
“22,23 Collectively, these tools con-fi rmed that the included evidence base was suf fi ciently robust to support an integrated synthesis of leadership styles and digital health outcomes within the Ethiopian PHC context.”
Ouédraogo, H. S., Ngangue, P. A. G., Kaboré, A., et al. (2026) · BMC Family Practice · doi:10.1186/s12875-026-03264-y
“The methodological rigour of the included studies was evaluated using an adapted version of the Mixed Meth-ods Assessment Tool (MMAT) [37, 38].”
Machine Learning na Gestão de Recursos Humanos em Saúde: Uma Revisão Sistemática
Boas, A. C. V., Sendim, F., Costa, F., et al. (2026) · TER ISAVE · doi:10.71399/hxkd3a26
Ndlovu, V. A., Ntho, T. A., Mbombi, M. O., & Ngwenya, M. W. (2026) · Journal of Public Health in Africa · doi:10.4102/jphia.v17i1.1600
“22 This appraisal tool was selected for its established validity and reliability.”
Müller, P., Veit, R., Hofstetter, S., & Jahn, P. (2026) · JMIR Nursing · doi:10.2196/84118 · Cited by 2
“The methodological rigor of the study was ensured by adhering to the Good Reporting of a Mixed Methods Study (GRAMMS) checklist [47] and the Mixed Methods Appraisal Tool (MMAT) [48].”
Peng, Y., Alias, B. S., Wan, X., & Mansor, A. N. (2026) · Discover Sustainability · doi:10.1007/s43621-026-03011-4 · Cited by 9
Ubuntu meets algorithms: Indigenous knowledge systems as a foundation for ethical AI
Mugunzva, F. I., & Rankhumise, E. M. (2026) · Strategic Business Research · doi:10.1016/j.sbr.2026.100119 · Cited by 1
Bradley, R., Conway, A., Ryan, A. A., Muldrew, D. H. L., & Penney, S. (2026) · doi:10.2196/preprints.94973
Adonteng‐Kissi, B., Wang, J., Zong, L., et al. (2026) · Social Work Education · doi:10.1080/02615479.2026.2646935
Hawkins, R. L., Craveiro, J., Kellar, I., & Brown, J. V. E. (2026) · BMC Health Services Research · doi:10.1186/s12913-026-14305-8 · Cited by 1
“Overall methodological quality of studies, assessed using the 2018 version of the MMAT [30], was varied.”
Metadata
- Authors
- Hong, Q. N., Fàbregues, S., Bartlett, G., Boardman, F., Cargo, M., Dagenais, P., Gagnon, M.-P., Griffiths, F., Nicolau, B., O’Cathain, A., Rousseau, M.-C., Vedel, I., & Pluye, P.
- Affiliations
- McGill University (CA)
- Universitat Oberta de Catalunya (ES)
- University of Warwick (GB)
- University of Canberra (AU)
- Université de Sherbrooke (CA)
- Université Laval (CA)
- University of Sheffield (GB)
- Institut National de la Recherche Scientifique (CA)
- Corresponding author
- Pluye, Pierre
- ORCID
- Venue
- Education for Information
- Publisher
- SAGE Publications
- Published
- 2018-12-18
- Volume
- 34
- Issue
- 4
- Pages
- 285-291
- ISSN
- 0167-8329 (ISSN-L)
- 1875-8649
- OpenAlex
- W2901724277
- Cited by
- 4,737
- References
- 21
- Citations by year
- 2026: 1312 · 2025: 974 · 2024: 800 · 2023: 610 · 2022: 501 · 2021: 370 · 2020: 143 · 2019: 22 · 2018: 3
- FWCI
- 102.21
- Type
- journal article
- Language
- en
- Open access
- green
- hdl.handle.net
- Licence
- cc-by-nc-nd
- Locations
- Education for Information (publishedVersion)
- O2 - Repositori Institucional (Universitat Oberta de Catalunya) (publishedVersion, cc-by-nc-nd)
- RECERCAT (Consorci de Serveis Universitaris de Catalunya) (publishedVersion, cc-by-nc-nd)
- White Rose Research Online (University of Leeds, The University of Sheffield, University of York) (acceptedVersion)
- Warwick Research Archive Portal (University of Warwick) (acceptedVersion)
- HAL (submittedVersion)
- eScholarship@McGill (McGill) (submittedVersion)
- LA Referencia (Red Federada de Repositorios Institucionales de Publicaciones Científicas) (submittedVersion, cc-by-nc-nd)
- eScholarship@McGill (McGill) (submittedVersion, cc-by-nc-nd)
- EspaceINRS (National Institute for Scientific Research (Canada)) (submittedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Sociology and Political Science › Delphi Technique in Research
- Social Sciences › Education › Qualitative Research Methods and Applications
- Keywords
- Mixed Methods Appraisal Tool, critical appraisal, study quality assessment, mixed methods, systematic reviews, qualitative research, quantitative research, randomized controlled trials, non-randomized studies, e-Delphi, inter-rater reliability
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.