The Mixed Methods Appraisal Tool (MMAT) version 2018 for information professionals and researchers
Open accessThe DOI matches its Crossref record; 21 of the 21 references it cites were found in a registry or an index; 2 of the 26 sources its claims cite were checked for support.
Abstract
INTRODUCTION: Appraising the quality of studies included in systematic reviews combining qualitative and quantitative evidence is challenging. To address this challenge, a critical appraisal tool was developed: the Mixed Methods Appraisal Tool (MMAT). The aim of this paper is to present the enhance ments made to the MMAT. DEVELOPMENT: The MMAT was initially developed in 2006 based on a literature review on systematic reviews combining qualitative and quantitative evidence. It was subject to pilot and interrater reliability testing. A revised version of the MMAT was developed in 2018 based on the results from usefulness testing, a literature review on critical appraisal tools and a modified e-Delphi study with methodological experts to identify core criteria. TOOL DESCRIPTION: The MMAT assesses the quality of qualitative, quantitative, and mixed methods studies. It focuses on methodological criteria and includes five core quality criteria for each of the following five categories of study designs: (a) qualitative, (b) randomized controlled, (c) nonrandomized, (d) quantitative descriptive, and (e) mixed methods. CONCLUSION: The MMAT is a unique tool that can be used to appraise the quality of different study designs. Also, by limiting to core criteria, the MMAT can provide a more efficient appraisal.
Overview
CoChat Summary
TL;DR
This paper presents version 2018 of the Mixed Methods Appraisal Tool (MMAT), a single critical appraisal tool for assessing the methodological quality of qualitative, quantitative, and mixed methods studies in mixed studies reviews. It describes the tool's 25 criteria across five study design categories, its development history, and guidance for use.
Key findings
The MMAT version 2018 includes a total of 25 criteria and 2 screening questions, covering five categories of study designs: (a) qualitative, (b) randomized controlled trial, (c) nonrandomized, (d) quantitative descriptive, and (e) mixed methods, with five core criteria per category rated as yes, no, or can't tell. The tool was developed from a 2006 first version (15 criteria across four categories), revised in 2011, and further revised into the 2018 version based on usefulness testing with 20 researchers, a literature review on critical appraisal tools, and a modified e-Delphi study with 73 methodological experts. Two interrater reliability studies used 32 and 261 papers respectively and showed the need to clarify criteria, particularly for nonrandomized and qualitative studies. The paper argues the MMAT offers a time-efficient alternative by limiting to core criteria and uniquely includes specific criteria for mixed methods studies, though its criteria are harder to judge because they target methodological rather than reporting quality.
Methods
This is a tool description paper (third in a three-part series on systematic reviews for information professionals) rather than an empirical study; it narrates the development of the MMAT and explains how to apply it. Development drew on a literature review on mixed studies reviews, pilot testing in workshops with four reviewers appraising six studies, two interrater reliability studies (32 and 261 papers), usefulness testing via interviews with 20 researchers, a literature review on critical appraisal tools, and a modified e-Delphi study with 73 methodological experts. Use involves three steps: two optional screening questions, selection of the appropriate design category (or categories), and rating each criterion as yes, no, or can't tell, with detailed presentation of ratings advised over an overall numerical score.
Limitations
The paper states that revision of the MMAT has so far focused on content validity and usefulness, and that further testing on its validity and reliability will be needed; it also notes that modifications may be necessary as evidence develops. It states that MMAT criteria are more difficult to judge than other appraisal tools because they focus on methodological quality rather than reporting quality. Evident from the text: the tool's development evidence base is described only in summary form, with reliability and validity results reported by reference to prior studies rather than presented here.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
2 Valid · 0 Not valid · 18 Not checked · 11 Own results and interpretations
Every source the paper cites for this claim was checked and supports it.
In the paper
“To further its development, usefulness testing was performed by interviewing 20 researchers who have used the MMAT in a systematic review or have contacted the developer of the tool for questions or permission to use the tool (18).”
What the sources say
- Supportedcites [18] 18. Hong QN, Gonzalez-Reyes A, Pluye P. Improving the usefulness of a tool for appraising the quality of qualitative, quantitative and mixed methods studies, the Mixed Methods Appraisal Tool (MMAT). J Eval Clin Pract. 2018;24(3):459-67. source
The Abstract reports semi-structured interviews with 20 participants to explore the MMAT’s usefulness. The Study Participants section says researchers were recruited through systematic reviews that used the MMAT and a list of people who had contacted its developer for permission or guidance. (LLM, conf=0.94)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“The results of this study were helpful to identify changes to be made in the MMAT.”
What the sources say
- Supportedcites [18] 18. Hong QN, Gonzalez-Reyes A, Pluye P. Improving the usefulness of a tool for appraising the quality of qualitative, quantitative and mixed methods studies, the Mixed Methods Appraisal Tool (MMAT). J Eval Clin Pract. 2018;24(3):459-67. source
The Discussion says the study explored MMAT users’ experiences to help revise the tool and identified themes suggesting areas for improvement. The Recommendations for Improving the MMAT then proposes specific changes on the basis of those results. (LLM, conf=0.94)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
In the paper
“Moreover, it includes specific criteria for mixed methods studies, which is not often found in other tools (21).”
What the sources say
- Not matchedcites [21] 21. Pluye P. Critical appraisal tools for assessing the methodological quality of qualitative, quantitative and mixed methods studies included in systematic mixed studies reviews. J Eval Clin Pract. 2013;19(4):722.
We searched by its title and found no matching paper.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Not matched
In the paper
“To address this challenge, a critical appraisal tool for assessing the quality of quantitative, qualitative and mixed methods studies was developed: the Mixed Methods Appraisal Tool (MMAT) (7).”
What the sources say
- Not checked — no full text of the sourcecites [7] 7. Pluye P, Gagnon MP, Griffiths F, Johnson-Lafleur J. A scoring system for appraising mixed methods research, and concomitantly appraising qualitative, quantitative and mixed methods primary studies in mixed studies reviews. Int J Nurs Stud. 2009;46(4):529-46. source
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“Moreover, it includes specific criteria for mixed methods studies, which is not often found in other tools (21).”
What the sources say
- Not matchedcites [21] 21. Pluye P. Critical appraisal tools for assessing the methodological quality of qualitative, quantitative and mixed methods studies included in systematic mixed studies reviews. J Eval Clin Pract. 2013;19(4):722.
We searched by its title and found no matching paper.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Not matched
5 more claims
Create a free account to see all 31 claims and to verify the 1 reference still open.
- It may state two separate claims as one. 7
- Probably not a substantial claim. 16
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Quality was assessed using two validated tools by study design. Quantitative studies were appraised using the Mixed Methods Appraisal Tool Version 2018, which evaluates five methodological criteria on a binary scale (Hong et al., 2018). Qualitative studies were additionally appraised using the Critical Appraisal Skills Programme Qualitative Checklist, which evaluates 10 criteria (CASP UK, 2018).”
From Cultural stigma and caregiver burden among family caregivers of people with nervous system disorders in the Middle East and North Africa: A mixed-methods systematic review (2026) · cites this paper as Hong et al., 2018
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“The quality of the included studies was assessed using the 2018 version of the Mixed Methods Appraisal Tool (MMAT).15”
From Psychosocial interventions for siblings of children with cancer: A mixed methods systematic review (2021) · cites this paper as [15]
The cited source says what the claim attributes to it.
In the citing paper
“To further limit risks of bias, a formal risk of bias assessment tool (ROB-tool), the Mixed Methods Appraisal Tool [52], was used to evaluate the overall study quality.”
From Mediators of outcome in adolescent psychotherapy and their implications for theories and mechanisms of change: a systematic review (2024) · cites this paper as [52]
The cited source says what the claim attributes to it.
In the citing paper
“LM, with the support of a research assistant, assessed the methodological quality of empirical studies (qualitative, quantitative and mixed methods) using the Mixed Methods Appraisal Tool (MMAT) [45].”
From Transnationalism and care of migrant families during pregnancy, postpartum and early-childhood: an integrative review (2020) · cites this paper as [45]
The cited source says what the claim attributes to it.
In the citing paper
“The quality of the included studies was independently assessed by two authors using the Mixed Methods Appraisal Tool (MMAT) (Hong et al., 2018).Disagreements were resolved through discussion.”
From Gamification on Mathematics Engagement and Motivation in Secondary School and Higher Education: A Systematic Review and Meta-Analysis (2026) · cites this paper as Hong et al., 2018
29 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ 4. Conclusion · p. 5
“## 4. Conclusion”
§ 4. How to use the MMAT? · p. 4
“## 4. How to use the MMAT?”
Citations
Cites
Pace, R., Pluye, P., Bartlett, G., Macaulay, A. C., Salsberg, J., Jagosh, J., & Seller, R. (2012) · International Journal of Nursing Studies · doi:10.1016/j.ijnurstu.2011.07.002 · Cited by 1,331 · cited 2×
“Then, two interrater reliability studies on the MMAT were conducted using respectively 32 and 261 papers (Pace et al., 2012; Souto et al., 2015).”
Self-citationPluye, P., Gagnon, M.-P., Griffiths, F., & Johnson-Lafleur, J. (2009) · International Journal of Nursing Studies · doi:10.1016/j.ijnurstu.2009.01.009 · Cited by 1,119 · cited 3×
“This led to develop a first version of the MMAT that is in line with a social constructionist worldview (Pluye et al., 2009).”
InfluentialSelf-citationHong, Q. N., Gonzalez‐Reyes, A., & Pluye, P. (2018) · Journal of Evaluation in Clinical Practice · doi:10.1111/jep.12884 · Cited by 1,107 · cited 3×
“To further its development, usefulness testing was performed by interviewing 20 researchers who have used the MMAT in a systematic review or had contacted the developer of the tool for questions or permission to use the tool (Hong et al., 2018a).”
Self-citationSystems to rate the strength of scientific evidence.
West, S. L., King, V., Carey, T. S., Lohr, K. N., Mckoy, N. A., Sutton, S. F., & Lux, L. J. (2002) · PubMed Central · Cited by 923
Zaza, S., Wright-De Agüero, L. K., Briss, P. A., Truman, B. I., Hopkins, D. P., Hennessy, M. H., Sosin, D. M., Anderson, L., Carande-Kulis, V. G., Teutsch, S. M., et al. (2000) · American Journal of Preventive Medicine · doi:10.1016/s0749-3797(99)00122-1 · Cited by 464 · cited 1×
“The algorithm was developed based on several existing algorithms of quantitative study designs (Hartling et al., 2010; Hartling et al., 2011; National Institute for Health Care Excellence, 2012; Scottish Intercollegiate Guidelines Network; Seo et al., 2016; West et al., 2002; Zaza et al., 2000).”
Souto, R. Q., Khanassov, V., Hong, Q. N., Bush, P. L., Vedel, I., & Pluye, P. (2015) · International Journal of Nursing Studies · doi:10.1016/j.ijnurstu.2014.08.010 · Cited by 375 · cited 1×
“Then, two interrater reliability studies on the MMAT were conducted using respectively 32 and 261 papers (Pace et al., 2012; Souto et al., 2015).”
Self-citationA review of critical appraisal tools show they lack rigor: Alternative tool structure is proposed
Crowe, M., & Sheppard, L. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2010.02.008 · Cited by 261 · cited 2×
“Currently, there exist more than 500 critical appraisal tools (i.e., checklists including a list of criteria to judge the quality of a study) (Bai et al., 2012; Crowe & Sheppard, 2011; Heyvaert et al., 2013; Santiago-Delefosse et al., 2016).”
Carroll, C., Booth, A., & Lloyd-Jones, M. (2012) · Qualitative Health Research · doi:10.1177/1049732312452937 · Cited by 248 · cited 1×
“Methodological quality criteria are more difficult to interpret because the reviewers need to judge whether the reported results of a study are trustworthy (Carroll et al., 2012; Hong & Pluye, 2018a).”
Adjustment of meta-analyses on the basis of quality scores should be abandoned
Herbison, P., Hay-Smith, J., & Gillespie, W. J. (2006) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2006.03.008 · Cited by 207 · cited 1×
“Currently in the literature, it is discouraged to calculate an overall score from the ratings of each criterion (Herbison et al., 2006; Higgins & Green, 2008; Viswanathan et al., 2012).”
Critical Appraisal of Mixed Methods Studies
Heyvaert, M., Hannes, K., Maes, B., & Onghena, P. (2013) · Journal of Mixed Methods Research · doi:10.1177/1558689813479449 · Cited by 168 · cited 1×
“Currently, there exist more than 500 critical appraisal tools (i.e., checklists including a list of criteria to judge the quality of a study) (Bai et al., 2012; Crowe & Sheppard, 2011; Heyvaert et al., 2013; Santiago-Delefosse et al., 2016).”
Cited by
Items 4,001–4,100 of 4,578
Ethical challenges to patient autonomy in the era of artificial intelligence: a systematic review
Mohammadnejad, S., Raiesifar, A., Bazmi, S., & Ghonodi, F. (2026) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-026-03614-x · Cited by 2
“It assesses criteria such as appropriateness of the research question, data collection methods, and integration of qualitative and quantitative components [41].”
Tshilate, L., Amponsah-Dacosta, F., & Mhlongo, S. E. (2026) · Springer proceedings in earth and environmental sciences · doi:10.1007/978-3-032-27507-3_3
Selvaratnam, N. C., Koo, F. K., Ekanayake, K., & Arora, A. (2026) · BMC Public Health · doi:10.1186/s12889-026-27776-5 · Cited by 1
“The MMAT facilitates the assessment of the methodological quality of studies with diverse study designs, including mixed-methods study designs, which is often not observed in other tools [52, 53].”
From harassment to datafication: a systematic evidence synthesis of women’s cybersecurity risks
Kamarudin, S., Tang, Q., Firmantoro, V., Yustisia, I. R., & Wahid, A. (2026) · Journal of Information Communication and Ethics in Society · doi:10.1108/jices-12-2025-0423 · Cited by 2
Arbuckle, B., Forster, E., & Mitchell, A. E. (2026) · Quality of Life Research · doi:10.1007/s11136-026-04266-0 · Cited by 1
“The MMAT was used for quality appraisal of the included studies [24] (see online Appendix 1).”
Julius, N., Zhong, Y., Rajabzadeh, V., Racine, S., Hayhoe, B. W. J., Barber, S., & Otis, M. (2026) · doi:10.2196/preprints.90489
Lizárraga, C., Castillo‐Pérez, I., & Grindlay, A. L. (2026) · Women s Studies International Forum · doi:10.1016/j.wsif.2026.103353
Clarifying the concept of visual load: a systematic review
Wang, X., & Yu, R. (2026) · Ergonomics · doi:10.1080/00140139.2026.2678470
Jang, S. L., Kolahdooz, F., Zakkar, M. A., Sharma, R., Harris, R., Manning, C., Wagg, A., Corriveau, A., Irlbacher‐Fox, S., Tarrant, M., et al. (2026) · Women s Health · doi:10.1177/17455057261456875
Mamprin, C., Salvas, M., Gervais, C., Miljus, M., & Fontaine, N. M. G. (2026) · International Journal of Bullying Prevention · doi:10.1007/s42380-026-00364-2
Guallart, C. (2026) · South Florida Journal of Development · doi:10.46932/sfjdv7n6-004
Coteur, K., Tops, L., Bekkering, G. E., Fonseca, L. D. A. F., van den Brand, F. A., Cloetens, H., Zeng, L., Aertgeerts, B., Delvaux, N., Agarwal, A., et al. (2026) · BMJ Open · doi:10.1136/bmjopen-2025-109865
“Reviewers (LT, KC, LdAFF, FvdB and HC) used the Mixed Methods Appraisal Tool 26 to assess methodological quality of individual studies.”
Transformaciones narrativas en preescolar usando entornos digitales para el aprendizaje del inglés
Zúñiga, A. Y. G. (2025) · Revista Multidisciplinaria Voces de América y el Caribe · doi:10.69821/remuvac.v2i2.146
Albornoz, W. (2026) · Information Communication & Society · doi:10.1080/1369118x.2026.2680989
Digital Maturity in Integrated Care Systems: Development strategies – A Scoping Review (Preprint)Preprint
Vilas-Boas, P., Seringa, J., Abreu, M., & Magalhães, T. (2026) · doi:10.2196/preprints.103869
DAHOU, Y., Idrissi, Y. E. B. E., & Khaddar, M. A. E. (2026) · doi:10.2196/preprints.103963
Asiedu, C., Ahiagbede, D. K., Johnson, E., BILSON, S. E., Atitsogbey, G., & Asiedu, E. F. O. (2026) · SAGE Open Medicine · doi:10.1177/20503121261459803 · Cited by 1
“For Mixed Method Appraisal Tool (MMAT), 15 1–2 (High risk of bias), 3–4 (Moderate risk of bias), and 5 (Low risk of bias).”
Higher education and quality assurance: coping mechanism in the age of AI
Zaky, H. (2026) · SN Social Sciences · doi:10.1007/s43545-026-01511-8 · Cited by 1
“Consistent with the PRISMA 2020 guidelines (Page et al. 2021 ) for systematic reviews that incorporate heterogeneous study designs, each of the 84 retained cor - pus articles was appraised for methodological quality using the Mixed Methods Appraisal Tool (MMAT; Hong et al. 2018).”
InfluentialAssessing mental fatigue in football: a systematic review
Aveiro, D., Campos, F., Rodrigues, R. N., Martins, F., & Kiely, J. (2026) · Frontiers in Sports and Active Living · doi:10.3389/fspor.2026.1861469
Exploring Sport Coaches’ Experiences of Online Learning Since 2017: A Systematic Review
Bush, J., Roberts, S. J., Twitchen, A., & Cronin, C. (2026) · International Sport Coaching Journal · doi:10.1123/iscj.2025-0034
Htay, Y. M. H., Foster, M., & Mowat, R. M. (2026) · Journal of Pediatric Nursing · doi:10.1016/j.pedn.2026.05.044
Best strategies to teach nutrition courses in graduate programs: a systematic review
Hojeij, Z., Haidar, S. A., Rizk, R., et al. (2026) · Frontiers in Education · doi:10.3389/feduc.2026.1761722
Cardoso, G. T. D. S., Abreu, H. M. B. F., Lacerda, H. C., & Suassuna, M. A. F. M. (2026) · Revista interdisciplinar em saúde · doi:10.35621/23587490.v13.n1.p1488-1497
Kormas, S., & Andriopoulos, P. (2026) · Clinical eHealth · doi:10.1016/j.ceh.2026.06.001
Colom-Rosselló, M., Abbate, M., Capitán‐Moyano, L., Pastor-Ramon, E., Fernández, M. E., Yañez, A. M., & Bennasar‐Veny, M. (2026) · Campbell Systematic Reviews · doi:10.1177/18911803261458193
Prasetia, A. R., & Fauza, N. A. (2025) · Novatio Journal of Management Technology and Innovation · doi:10.61978/novatio.v3i4.1483 · Cited by 1
“…adapted from established mixed-method and review-quality guidance, with emphasis on design clarity, sampling adequacy, transparency of construct operationalization, appropriateness of analytical technique, and sufficiency of reported findings for synthesis (Hong et al., 2018; Simsek et al., 2025).”
Technology-Driven Globalization of Higher Education
Noche, E. B. (2026) · IGI Global eBooks · doi:10.4018/979-8-3373-8267-8.ch014
Akhtar, T. A., & Stephenson, Z. (2026) · Journal of Forensic Psychiatry and Psychology · doi:10.1080/14789949.2026.2686363
Boocock, R. C., Jones, C., & Lake, A. A. (2026) · Journal of Human Nutrition and Dietetics · doi:10.1111/jhn.70287
Telerehabilitation in children and adolescents with intellectual disability: a systematic review
Micai, M., Gila, L., Caruso, A., Morelli, D., Totino, M. G., Balboni, G., Belacchi, C., Colucci, A., Fantini, C., Metitieri, T. T., et al. (2026) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2026.1855260
Dzramado, V. L., Amoah, W. W., Antwi, S., et al. (2026) · Frontiers in Public Health · doi:10.3389/fpubh.2026.1803063
Vovk, Y., Vovk, O., Голотенко, О. С., Vovk, Y., & Sipravska, M. (2026) · Journal of Sustainable Development of Transport and Logistics · doi:10.14254/jsdtl.2026.11-1.03 · Cited by 1
Bone maceration: A systematic review of techniques, efficiency and structural outcomes
Mandler, E. M., Mittendorfer, J. M., & Hirtler, L. (2026) · Annals of Anatomy - Anatomischer Anzeiger · doi:10.1016/j.aanat.2026.152866
Williams, E., Horton, R., Mulholland, T., & Roper, L. (2026) · doi:10.2196/preprints.104493
Cannabis Use Motives Across the Reproductive Continuum: An Integrative Literature Review
Roland, A. (2026) · Cannabis · doi:10.26828/cannabis/2026/000363
“Then, study quality was evaluated with an appropriate quality assessment tool for each study design, including the Mixed Methods Appraisal Tool (Hong et al., 2018), the Scale for the Quality Assessment of Narrative Reviews (Baethge et al., 2019), and the Joanna Briggs Institute critical appraisal tools (Aromataris et…”
How is housing insecurity measured among older adults? A systematic review
Cai, T., Peng, C., Lam, S. Y., & Lu, P. (2026) · The Gerontologist · doi:10.1093/geront/gnag130
“…studies was assessed using the Mixed Methods Appraisal Tool (MMAT, version 2018), a critical tool developed for the appraisal stage of systematic reviews that encompass diverse study designs, including qualitative, quantitative, and mixed methods studies (Hong et al., 2018; Pluye et al., 2009).”
Rahman, A., & Wibisono, E. (2026) · Circular Economy and Sustainability · doi:10.1007/s43615-026-01031-w · Cited by 3
“The MMAT was selected given the structured and flexible framework suitable for systematic reviews designed for qualita - tive, quantitative, and mixed-methods studies [24], consistent with qualitative and social-science evidence synthesis guidance.”
Age-Friendly Housing Models and Frameworks: A Systematic Review
Sanchaniya, R. J., Geipele, I., Kundziņa, A., & Černeckienė, J. (2026) · Real Estate Management and Valuation · doi:10.2478/remav-2026-0014
“The Mixed Methods Appraisal Tool (Hong et al., 2018) was employed to evaluate the credibility of empirical studies, facilitating cross-design comparability among qualitative, quantitative, and mixed-methods evidence.”
Eyrenci, A., Ertuna, L., & Bahçivan, O. (2026) · BMC Psychology · doi:10.1186/s40359-026-04915-6
“The methodological quality of the study was evaluated against the criteria of the Mixed Methods Appraisal Tool (MMAT), version 2018 [25].”
Eixo imunogenético no transtorno do espectro autista:CHD8, FOXP1 e inflamassoma NLRP3 em uma revisão
de Oliveira, J. A., Saldivias, R. C. D. C., & Baminger, J. (2026) · Research Society and Development · doi:10.33448/rsd-v15i6.50906
Bouzid, M., Guerrero, W. J., Torres‐Guevara, L. E., et al. (2026) · The Extractive Industries and Society · doi:10.1016/j.exis.2026.101984
Wang, Y., & Lan, Y. (2026) · Frontiers in Psychology · doi:10.3389/fpsyg.2026.1866711
Changing organizational culture in community sport: a systematic review
Woodforde, J., Thomas, G., Harrison, Z., Mulcahy, S., Townsend, S., & Cairney, J. (2026) · Frontiers in Sports and Active Living · doi:10.3389/fspor.2026.1852179 · Cited by 2
“Rather than relying on a single composite score, the MMAT encourages reporting on each criterion to allow more nuanced interpretation of methodological strengths and limitations.”
InfluentialAgula, C., Kanmiki, E. W., Ofori, C. A. E., Ogunoye, O. R., Church, K., Ouédraogo, R., & Bawah, A. A. (2026) · BMJ Open · doi:10.1136/bmjopen-2025-113986
“26 Each study will be evaluated independently by two reviewers (CA and EWK) using the five main domains of MMAT: (1) Clarity of the research question, (2) Appro-priateness of the design employed by the study, (3) Rele-vance of the sampling strategy and data sources, (4) Quality of measurements or…”
Chimatiro, C. S., Mianda, S., Hajison, P., et al. (2026) · PLoS ONE · doi:10.1371/journal.pone.0349448
“The Mixed Method Appraisal Tool (MMAT) version 2018 was used to evaluate the quality of the selected articles [38].”
Wang, M., Akhter, S., & Tan, M. (2026) · Frontiers in Psychology · doi:10.3389/fpsyg.2026.1815314 · Cited by 1
“Among mixed-methods studies ( n = 2), both met MMAT screening criteria and were rated adequate on domain-specific items.”
InfluentialSystematic Review of Interventions for Twice-Exceptional Autistic Learners
Radhakrishnan, A., Buckley, C., O’Reilly, C., Pagkratidou, M., Stack, N., & Boran, L. (2026) · Education Sciences · doi:10.3390/educsci16060941 · Cited by 3
Mental workload in the maritime industry: a pre-registered systematic review of influencing factors
Staab, V., Ramrath, M., Bačkalov, I., Athanassiou, G., & Liebherr, M. (2026) · WMU Journal of Maritime Affairs · doi:10.1007/s13437-026-00420-0
Ghumman, S. S., Soi, P., & Shirmohammadlou, M. (2026) · Applied Psychology · doi:10.1111/apps.70112
Zhang, S. S., Hassan, N. C., Hassan, R., & Wang, C. (2025) · The Pertanika Journal of Learning Pedagogy and Educational Leadership · doi:10.47836/pjlpel.1.1.04
Kaur, M., Teo, C. H., Jefnee, M. A. I., Kaur, S., Min, M. T. K., Munusamy, T., Ang, T. F., & Ganesan, D. (2026) · Patient Education and Counseling · doi:10.1016/j.pec.2026.109750
Contraceptive Decision-Making Among Muslim Women in Bangladesh: A Systematic ReviewPreprint
Prima, F. J., & Thamminaina, A. (2026) · Research Square · doi:10.21203/rs.3.rs-9789342/v1
Li, C., Du, J., & Mohammadpourkarbasi, H. (2026) · Indoor Air · doi:10.1155/ina/9340794
Dewi, Y. A., Fawzia, A. A., Artina, N. R., Rifaldi, A., Andriyaningrum, R., Tantiana, R., & Hapsari, G. P. (2026) · F1000Research · doi:10.12688/f1000research.183110.1
Recovery‐Oriented Conversations in Psychiatric Care: An Integrated Systematic Review
Ekezie, P. E., Rozental, A., Looi, G. E., Werneke, U., & Gabrielsson, S. (2026) · International Journal of Mental Health Nursing · doi:10.1111/inm.70295
A Rapid Review of Service Configurations and Models of Care for People With Young Onset Dementia
Mulhall, S., Gibson, D. M., Cations, M., Loi, S. M., Isbel, S. T., Northey, J. M., Bail, K. S., & D'Cunha, N. M. (2026) · Health & Social Care in the Community · doi:10.1155/hsc/5657771
STEMing evidence-based study strategies: a systematic review of literature
Yuriev, E., Chen, S., Vo, K., Terrill, A. E., Gomes, J., Kyne, S. H., & Sarkar, M. (2026) · International Journal of STEM Education · doi:10.1186/s40594-026-00632-z
“As the selected studies involved quantitative, qualitative, or mixed-methods approaches, the Mixed Method Appraisal Tool (MMAT) was used as an appraisal tool (Hong et al., 2018).”
Buckley, D., Mulvenna, M. D., Ennis, E., Bond, R., Tariq, Z., & Millman, C. M. (2026) · doi:10.2196/preprints.105256
Qadri, U., Muhafidin, D., Widianingsih, I., et al. (2026) · Research Square · doi:10.21203/rs.3.rs-10103777/v1
DINATA, Y. Z., & Pandin, M. G. R. (2026) · Research Square · doi:10.21203/rs.3.rs-10120180/v1
Gajbhiye, R., Kedar, P. S., Rani, P., Chandekar, S., Nagaraj, J., & Madkaikar, M. R. (2026) · Journal of Community Genetics · doi:10.1007/s12687-026-00912-9 · Cited by 1
“Although methodological quality appraisal is not required for scoping reviews, we used the Mixed Methods Appraisal Tool (MMAT) (Hong et al. 2018 ) and Joanna Briggs Insti - tute (JBI) (McArthur et al. 2025) critical appraisal check-lists to contextualise the strengths and limitations of the included studies (Table S2…”
Asamoah, P. E., & Osei, A. (2026) · Journal of Public Health in Africa · doi:10.4102/jphia.v17i1.1820
“The Mixed-Methods Appraisal Tool 17 was used because it accommodates the qualitative, quantitative descriptive and mixed-methods designs represented in the review.”
Luin, J. A., & Lahindah, L. (2026) · International Journal of Evidence Based Coaching and Mentoring · doi:10.1177/17418305261455111
“The methodological quality of the included studies was assessed using the Mixed Methods Appraisal Tool (MMAT), selected for its capacity to evaluate diverse research designs in leadership development literature (Hong et al., 2018) while ensuring alignment with servant leadership review standards.”
InfluentialHaryadi, D., Yensy, N. A., Wahyuni, S. T., Nurhidayati, E., & Danim, S. (2026) · Learning in Context · doi:10.1016/j.lecon.2026.100029
Wang, Y., Zhao, L., Zou, F., Zhou, H., Li, X., Li, C., & Linyan, X. (2026) · PLoS ONE · doi:10.1371/journal.pone.0347798
“However, to provide context for interpreting findings, methodological limitations of included studies will be descriptively characterized using the Mixed Methods Appraisal Tool (MMAT) [31].”
Achieng, M. S. (2026) · Frontiers in Sustainable Food Systems · doi:10.3389/fsufs.2026.1847230 · Cited by 1
The acceptability of wearable devices for promoting adolescent mental wellbeing: A systematic review
Laiti, J., Rahul, R., Mundadan, M., Byrne, E., & Dunne, P. J. (2026) · Digital Health · doi:10.1177/20552076261450372 · Cited by 1
“The 2018 version of the Mixed Methods Appraisal Tool (MMAT) 25 was used to assess the quality of the included papers in this review.”
InfluentialMaotoana, M., Phalane, K., Tlowane, M., et al. (2026) · Frontiers in Education · doi:10.3389/feduc.2026.1740146
Vickery, T., Fisher, G., Pagano, L., Foo, D., & Braithwaite, J. (2026) · Campbell Systematic Reviews · doi:10.1177/18911803261460864
“The quality of articles will be evaluated using the Mixed-Methods Appraisal Tool (Hong et al., 2018), widely used for systematic reviews (Hong et al., 2018).”
Dlamini, B. I., Lawa, E., & Shange, H. S. (2026) · Frontiers in Education · doi:10.3389/feduc.2026.1872154
Hu, K. C., Raman, A. L., & Shan, F. Y. (2026) · Journal of Digital Educational Technology · doi:10.29333/jdet/18857
“Therefore, the Mixed Methods Appraisal Tool (MMAT) was selected and implemented within EPPI-Reviewer, as it enables the concurrent and standardized assessment of diverse methodological approaches under a unified framework (Hong et al., 2018).”
InfluentialWojtowicz, E., Yous, M., Baxter, P. E., & Kaasalainen, S. (2026) · Palliative Care and Social Practice · doi:10.1177/26323524261462628
Abbas, A. F., Mas’od, A., & Salleh, N. Z. M. (2026) · Current Psychology · doi:10.1007/s12144-026-09646-w
“Effective framing can create cognitive dissonance, reshape social narratives, and stimulate behavioral reconsideration (Grewal et al., 2025; Smith, 2020 ; do Canto et al., 2023 ; Garg et al., 2021 ).”
Systematic Review of Sponge City Indicators for Constructing a Thailand-Specific Sponginess Index
Thamakaison, S., & Positlimpakul, K. (2026) · IOP Conference Series Earth and Environmental Science · doi:10.1088/1755-1315/1629/1/012020
Nulft, V. V. D., Brinkman‐Stoppelenburg, A., van Vliet, L., van der Steen, J. T., & van der Linden, Y. M. (2026) · Journal of Alzheimer s Disease · doi:10.1177/13872877261461216 · Cited by 1
“42 One quantitative study received a low score (i.e., 2/5) due to differences between the control and intervention groups, incompleteness of the data, and not accounting for confounding factors.”
Virtual Reality and Arabic Language Learning: A Global Systematic Literature Review
Ziafia, R. K. A. F., Ahsanuddin, M., Fauzi, M. F., et al. (2026) · An-Nabighoh Jurnal Pendidikan dan Pembelajaran Bahasa Arab · doi:10.32332/an-nabighoh.v28i1.253-276
Li, Z., & Bohndick, C. (2026) · Educational Research Review · doi:10.1016/j.edurev.2026.100810 · Cited by 1
Psychological interventions for smoking cessation in pregnant women: a systematic review
Posse, C. M., & Míguez, M. C. (2026) · Nicotine & Tobacco Research · doi:10.1093/ntr/ntag143 · Cited by 1
Issah, M., & Adam, I. O. (2026) · Strategic Business Research · doi:10.1016/j.sbr.2026.100215
Luo, Y., Xiao, Q., Li, B., Zhang, Y., Liu, Y., Zhi, C., Zhang, Q., Li, M., & Ma, H. (2026) · Frontiers in Public Health · doi:10.3389/fpubh.2026.1887736
Semere, F. A., & Csíkos, C. (2026) · Education Sciences · doi:10.3390/educsci16071029 · Cited by 1
Enhancing climate-crop models for actionable agricultural climate change adaptation
Oluwatimilehin, I. A., Ayanlade, O. S., Olaitan, R. A., & Ayanlade, A. (2026) · Critical Insights in Climate Change · doi:10.1080/29931495.2026.2682696 · Cited by 1
Teachers’ STEAM competencies for the 21st century: A systematic review
Jingwen, C., Hidayat, R., & bin Hassan, M. Z. (2026) · Eurasia Journal of Mathematics Science and Technology Education · doi:10.29333/ejmste/18868 · Cited by 1
“This tool is suitable for evaluating a variety of research designs, including qualitative, quantitative, and mixed-methods studies (Hong et al., 2018; Utami et la., 2024).”
Kennard, A., Kılıç, A., Alsugeir, D. M., Hamada, Y., & Rangaka, M. X. (2026) · BMC Public Health · doi:10.1186/s12889-026-28299-9
“Two experienced independent reviewers (DA & AKi) assessed the quality of the included papers by using the Mixed Methods Appraisal Tool (MMAT) [28] and 10% the included papers were randomly selected to calibrate between reviewers.”
Jebbeh, R. A., & Alnaeem, M. M. (2026) · Applied Nursing Research · doi:10.1016/j.apnr.2026.152113 · Cited by 1
Self-management in Korean immigrants with type 2 diabetes in the United States: a systematic review
Kim, J. E., Jung, S. O., & Park, S. (2026) · Journal of Diabetes & Metabolic Disorders · doi:10.1007/s40200-026-01966-9
Ali, K., & Anagnostou, G. (2026) · Advances in Physical Education · doi:10.4236/ape.2026.163003
Mapping in-service teacher AI literacy: A systematic review of empirical studies
Kokoç, M., Ağırman, Ş., & Yağmurlugil, B. N. (2026) · Teaching and Teacher Education · doi:10.1016/j.tate.2026.105707 · Cited by 10
Medveyuk, S., Vervaecke, D., Nguyen, H., Corbett, K., Dymchuk, E., Baghini, B. Z., Slavescu, I., Tate, K., Hogan, D. B., Wagg, A., et al. (2026) · Research Square · doi:10.21203/rs.3.rs-9918025/v1
Alker, T., Hine, K., McKillop, N., & Prenzler, T. (2026) · Journal of Police and Criminal Psychology · doi:10.1007/s11896-026-09829-y · Cited by 1
Is Algorithm the Culprit of Information Cocoons: A Systematic Literature Review
Liu, Y., Zhao, H., & Lin, Y. (2026) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2026.2688495
Zhao, J., Shah, S. S. A., & Hamid, S. H. A. (2026) · Frontiers in Education · doi:10.3389/feduc.2026.1759097
Chichava, A. D., Zaqueu, L. C. M. A., & Sitoe, M. D. (2026) · Research Square · doi:10.21203/rs.3.rs-10203525/v1
Ng, R.-K., Ng, R. S., Tan, Q. Q. C., Beh, W. T., Wong, S. N., & Grundy‐Warr, C. (2026) · BMJ Public Health · doi:10.1136/bmjph-2025-003817 · Cited by 1
Nzowa, P. G., Elisa, N., Mungulluh, F., Mkilia, E. L., Malombe, H. H., Minja, G. G., Mambile, C., & Mwogosi, A. (2026) · SAGE Open · doi:10.1177/21582440261463355
Mesce, M., Scalzeri, M., Galvez-Sánchez, C. M., Camacho-Ruiz, J. A., Gras, R. M. L., & Galli, F. (2026) · Psychiatry and Clinical Neurosciences · doi:10.1111/pcn.70099
“44”
Research gaps concerning Blended Project-Based Learning in Computing Education
Ahmed, A. R., & Kolås, L. (2026) · International Conference on Information Technology Based Higher Education and Training (ITHET) · doi:10.1109/ithet69978.2026.11584954
“The Mixed Methods Appraisal Tool (MMAT) [12] was used to assess all 70 empirical studies, including quantitative, qualitative, and mixed‑methods research.”
Edwards, D. J., Clark, E., Csontos, J. K., Dobbins, M., Gillen, E., Hounsome, J. C., Neil‐Sztramko, S., Lewis, R. A., Mann, M. K., & Prue, G. (2026) · Research Square · doi:10.21203/rs.3.rs-8724550/v3
Metadata
- Authors
- Hong, Q. N., Fàbregues, S., Bartlett, G., Boardman, F., Cargo, M., Dagenais, P., Gagnon, M.-P., Griffiths, F., Nicolau, B., O’Cathain, A., Rousseau, M.-C., Vedel, I., & Pluye, P.
- Affiliations
- McGill University (CA)
- Universitat Oberta de Catalunya (ES)
- University of Warwick (GB)
- University of Canberra (AU)
- Université de Sherbrooke (CA)
- Université Laval (CA)
- University of Sheffield (GB)
- Institut National de la Recherche Scientifique (CA)
- Corresponding author
- Pluye, Pierre
- ORCID
- Venue
- Education for Information
- Publisher
- SAGE Publications
- Published
- 2018-12-18
- Volume
- 34
- Issue
- 4
- Pages
- 285-291
- ISSN
- 0167-8329 (ISSN-L)
- 1875-8649
- OpenAlex
- W2901724277
- Cited by
- 4,737
- References
- 21
- Citations by year
- 2026: 1312 · 2025: 974 · 2024: 800 · 2023: 610 · 2022: 501 · 2021: 370 · 2020: 143 · 2019: 22 · 2018: 3
- FWCI
- 102.21
- Type
- journal article
- Language
- en
- Open access
- green
- hdl.handle.net
- Licence
- cc-by-nc-nd
- Locations
- Education for Information (publishedVersion)
- O2 - Repositori Institucional (Universitat Oberta de Catalunya) (publishedVersion, cc-by-nc-nd)
- RECERCAT (Consorci de Serveis Universitaris de Catalunya) (publishedVersion, cc-by-nc-nd)
- White Rose Research Online (University of Leeds, The University of Sheffield, University of York) (acceptedVersion)
- Warwick Research Archive Portal (University of Warwick) (acceptedVersion)
- HAL (submittedVersion)
- eScholarship@McGill (McGill) (submittedVersion)
- LA Referencia (Red Federada de Repositorios Institucionales de Publicaciones Científicas) (submittedVersion, cc-by-nc-nd)
- eScholarship@McGill (McGill) (submittedVersion, cc-by-nc-nd)
- EspaceINRS (National Institute for Scientific Research (Canada)) (submittedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Sociology and Political Science › Delphi Technique in Research
- Social Sciences › Education › Qualitative Research Methods and Applications
- Keywords
- Mixed Methods Appraisal Tool, critical appraisal, study quality assessment, mixed methods, systematic reviews, qualitative research, quantitative research, randomized controlled trials, non-randomized studies, e-Delphi, inter-rater reliability
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.