The Mixed Methods Appraisal Tool (MMAT) version 2018 for information professionals and researchers
Open accessThe DOI matches its Crossref record; 21 of the 21 references it cites were found in a registry or an index; 2 of the 26 sources its claims cite were checked for support.
Abstract
INTRODUCTION: Appraising the quality of studies included in systematic reviews combining qualitative and quantitative evidence is challenging. To address this challenge, a critical appraisal tool was developed: the Mixed Methods Appraisal Tool (MMAT). The aim of this paper is to present the enhance ments made to the MMAT. DEVELOPMENT: The MMAT was initially developed in 2006 based on a literature review on systematic reviews combining qualitative and quantitative evidence. It was subject to pilot and interrater reliability testing. A revised version of the MMAT was developed in 2018 based on the results from usefulness testing, a literature review on critical appraisal tools and a modified e-Delphi study with methodological experts to identify core criteria. TOOL DESCRIPTION: The MMAT assesses the quality of qualitative, quantitative, and mixed methods studies. It focuses on methodological criteria and includes five core quality criteria for each of the following five categories of study designs: (a) qualitative, (b) randomized controlled, (c) nonrandomized, (d) quantitative descriptive, and (e) mixed methods. CONCLUSION: The MMAT is a unique tool that can be used to appraise the quality of different study designs. Also, by limiting to core criteria, the MMAT can provide a more efficient appraisal.
Overview
CoChat Summary
TL;DR
This paper presents version 2018 of the Mixed Methods Appraisal Tool (MMAT), a single critical appraisal tool for assessing the methodological quality of qualitative, quantitative, and mixed methods studies in mixed studies reviews. It describes the tool's 25 criteria across five study design categories, its development history, and guidance for use.
Key findings
The MMAT version 2018 includes a total of 25 criteria and 2 screening questions, covering five categories of study designs: (a) qualitative, (b) randomized controlled trial, (c) nonrandomized, (d) quantitative descriptive, and (e) mixed methods, with five core criteria per category rated as yes, no, or can't tell. The tool was developed from a 2006 first version (15 criteria across four categories), revised in 2011, and further revised into the 2018 version based on usefulness testing with 20 researchers, a literature review on critical appraisal tools, and a modified e-Delphi study with 73 methodological experts. Two interrater reliability studies used 32 and 261 papers respectively and showed the need to clarify criteria, particularly for nonrandomized and qualitative studies. The paper argues the MMAT offers a time-efficient alternative by limiting to core criteria and uniquely includes specific criteria for mixed methods studies, though its criteria are harder to judge because they target methodological rather than reporting quality.
Methods
This is a tool description paper (third in a three-part series on systematic reviews for information professionals) rather than an empirical study; it narrates the development of the MMAT and explains how to apply it. Development drew on a literature review on mixed studies reviews, pilot testing in workshops with four reviewers appraising six studies, two interrater reliability studies (32 and 261 papers), usefulness testing via interviews with 20 researchers, a literature review on critical appraisal tools, and a modified e-Delphi study with 73 methodological experts. Use involves three steps: two optional screening questions, selection of the appropriate design category (or categories), and rating each criterion as yes, no, or can't tell, with detailed presentation of ratings advised over an overall numerical score.
Limitations
The paper states that revision of the MMAT has so far focused on content validity and usefulness, and that further testing on its validity and reliability will be needed; it also notes that modifications may be necessary as evidence develops. It states that MMAT criteria are more difficult to judge than other appraisal tools because they focus on methodological quality rather than reporting quality. Evident from the text: the tool's development evidence base is described only in summary form, with reliability and validity results reported by reference to prior studies rather than presented here.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
2 Valid · 0 Not valid · 18 Not checked · 11 Own results and interpretations
Every source the paper cites for this claim was checked and supports it.
In the paper
“To further its development, usefulness testing was performed by interviewing 20 researchers who have used the MMAT in a systematic review or have contacted the developer of the tool for questions or permission to use the tool (18).”
What the sources say
- Supportedcites [18] 18. Hong QN, Gonzalez-Reyes A, Pluye P. Improving the usefulness of a tool for appraising the quality of qualitative, quantitative and mixed methods studies, the Mixed Methods Appraisal Tool (MMAT). J Eval Clin Pract. 2018;24(3):459-67. source
The Abstract reports semi-structured interviews with 20 participants to explore the MMAT’s usefulness. The Study Participants section says researchers were recruited through systematic reviews that used the MMAT and a list of people who had contacted its developer for permission or guidance. (LLM, conf=0.94)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“The results of this study were helpful to identify changes to be made in the MMAT.”
What the sources say
- Supportedcites [18] 18. Hong QN, Gonzalez-Reyes A, Pluye P. Improving the usefulness of a tool for appraising the quality of qualitative, quantitative and mixed methods studies, the Mixed Methods Appraisal Tool (MMAT). J Eval Clin Pract. 2018;24(3):459-67. source
The Discussion says the study explored MMAT users’ experiences to help revise the tool and identified themes suggesting areas for improvement. The Recommendations for Improving the MMAT then proposes specific changes on the basis of those results. (LLM, conf=0.94)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
In the paper
“Moreover, it includes specific criteria for mixed methods studies, which is not often found in other tools (21).”
What the sources say
- Not matchedcites [21] 21. Pluye P. Critical appraisal tools for assessing the methodological quality of qualitative, quantitative and mixed methods studies included in systematic mixed studies reviews. J Eval Clin Pract. 2013;19(4):722.
We searched by its title and found no matching paper.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Not matched
In the paper
“To address this challenge, a critical appraisal tool for assessing the quality of quantitative, qualitative and mixed methods studies was developed: the Mixed Methods Appraisal Tool (MMAT) (7).”
What the sources say
- Not checked — no full text of the sourcecites [7] 7. Pluye P, Gagnon MP, Griffiths F, Johnson-Lafleur J. A scoring system for appraising mixed methods research, and concomitantly appraising qualitative, quantitative and mixed methods primary studies in mixed studies reviews. Int J Nurs Stud. 2009;46(4):529-46. source
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“Moreover, it includes specific criteria for mixed methods studies, which is not often found in other tools (21).”
What the sources say
- Not matchedcites [21] 21. Pluye P. Critical appraisal tools for assessing the methodological quality of qualitative, quantitative and mixed methods studies included in systematic mixed studies reviews. J Eval Clin Pract. 2013;19(4):722.
We searched by its title and found no matching paper.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Not matched
5 more claims
Create a free account to see all 31 claims and to verify the 1 reference still open.
- It may state two separate claims as one. 7
- Probably not a substantial claim. 16
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Quality was assessed using two validated tools by study design. Quantitative studies were appraised using the Mixed Methods Appraisal Tool Version 2018, which evaluates five methodological criteria on a binary scale (Hong et al., 2018). Qualitative studies were additionally appraised using the Critical Appraisal Skills Programme Qualitative Checklist, which evaluates 10 criteria (CASP UK, 2018).”
From Cultural stigma and caregiver burden among family caregivers of people with nervous system disorders in the Middle East and North Africa: A mixed-methods systematic review (2026) · cites this paper as Hong et al., 2018
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“The quality of the included studies was assessed using the 2018 version of the Mixed Methods Appraisal Tool (MMAT).15”
From Psychosocial interventions for siblings of children with cancer: A mixed methods systematic review (2021) · cites this paper as [15]
The cited source says what the claim attributes to it.
In the citing paper
“To further limit risks of bias, a formal risk of bias assessment tool (ROB-tool), the Mixed Methods Appraisal Tool [52], was used to evaluate the overall study quality.”
From Mediators of outcome in adolescent psychotherapy and their implications for theories and mechanisms of change: a systematic review (2024) · cites this paper as [52]
The cited source says what the claim attributes to it.
In the citing paper
“LM, with the support of a research assistant, assessed the methodological quality of empirical studies (qualitative, quantitative and mixed methods) using the Mixed Methods Appraisal Tool (MMAT) [45].”
From Transnationalism and care of migrant families during pregnancy, postpartum and early-childhood: an integrative review (2020) · cites this paper as [45]
The cited source says what the claim attributes to it.
In the citing paper
“The quality of the included studies was independently assessed by two authors using the Mixed Methods Appraisal Tool (MMAT) (Hong et al., 2018).Disagreements were resolved through discussion.”
From Gamification on Mathematics Engagement and Motivation in Secondary School and Higher Education: A Systematic Review and Meta-Analysis (2026) · cites this paper as Hong et al., 2018
29 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ 4. Conclusion · p. 5
“## 4. Conclusion”
§ 4. How to use the MMAT? · p. 4
“## 4. How to use the MMAT?”
Citations
Cites
Pace, R., Pluye, P., Bartlett, G., Macaulay, A. C., Salsberg, J., Jagosh, J., & Seller, R. (2012) · International Journal of Nursing Studies · doi:10.1016/j.ijnurstu.2011.07.002 · Cited by 1,331 · cited 2×
“Then, two interrater reliability studies on the MMAT were conducted using respectively 32 and 261 papers (Pace et al., 2012; Souto et al., 2015).”
Self-citationPluye, P., Gagnon, M.-P., Griffiths, F., & Johnson-Lafleur, J. (2009) · International Journal of Nursing Studies · doi:10.1016/j.ijnurstu.2009.01.009 · Cited by 1,119 · cited 3×
“This led to develop a first version of the MMAT that is in line with a social constructionist worldview (Pluye et al., 2009).”
InfluentialSelf-citationHong, Q. N., Gonzalez‐Reyes, A., & Pluye, P. (2018) · Journal of Evaluation in Clinical Practice · doi:10.1111/jep.12884 · Cited by 1,107 · cited 3×
“To further its development, usefulness testing was performed by interviewing 20 researchers who have used the MMAT in a systematic review or had contacted the developer of the tool for questions or permission to use the tool (Hong et al., 2018a).”
Self-citationSystems to rate the strength of scientific evidence.
West, S. L., King, V., Carey, T. S., Lohr, K. N., Mckoy, N. A., Sutton, S. F., & Lux, L. J. (2002) · PubMed Central · Cited by 923
Zaza, S., Wright-De Agüero, L. K., Briss, P. A., Truman, B. I., Hopkins, D. P., Hennessy, M. H., Sosin, D. M., Anderson, L., Carande-Kulis, V. G., Teutsch, S. M., et al. (2000) · American Journal of Preventive Medicine · doi:10.1016/s0749-3797(99)00122-1 · Cited by 464 · cited 1×
“The algorithm was developed based on several existing algorithms of quantitative study designs (Hartling et al., 2010; Hartling et al., 2011; National Institute for Health Care Excellence, 2012; Scottish Intercollegiate Guidelines Network; Seo et al., 2016; West et al., 2002; Zaza et al., 2000).”
Souto, R. Q., Khanassov, V., Hong, Q. N., Bush, P. L., Vedel, I., & Pluye, P. (2015) · International Journal of Nursing Studies · doi:10.1016/j.ijnurstu.2014.08.010 · Cited by 375 · cited 1×
“Then, two interrater reliability studies on the MMAT were conducted using respectively 32 and 261 papers (Pace et al., 2012; Souto et al., 2015).”
Self-citationA review of critical appraisal tools show they lack rigor: Alternative tool structure is proposed
Crowe, M., & Sheppard, L. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2010.02.008 · Cited by 261 · cited 2×
“Currently, there exist more than 500 critical appraisal tools (i.e., checklists including a list of criteria to judge the quality of a study) (Bai et al., 2012; Crowe & Sheppard, 2011; Heyvaert et al., 2013; Santiago-Delefosse et al., 2016).”
Carroll, C., Booth, A., & Lloyd-Jones, M. (2012) · Qualitative Health Research · doi:10.1177/1049732312452937 · Cited by 248 · cited 1×
“Methodological quality criteria are more difficult to interpret because the reviewers need to judge whether the reported results of a study are trustworthy (Carroll et al., 2012; Hong & Pluye, 2018a).”
Adjustment of meta-analyses on the basis of quality scores should be abandoned
Herbison, P., Hay-Smith, J., & Gillespie, W. J. (2006) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2006.03.008 · Cited by 207 · cited 1×
“Currently in the literature, it is discouraged to calculate an overall score from the ratings of each criterion (Herbison et al., 2006; Higgins & Green, 2008; Viswanathan et al., 2012).”
Critical Appraisal of Mixed Methods Studies
Heyvaert, M., Hannes, K., Maes, B., & Onghena, P. (2013) · Journal of Mixed Methods Research · doi:10.1177/1558689813479449 · Cited by 168 · cited 1×
“Currently, there exist more than 500 critical appraisal tools (i.e., checklists including a list of criteria to judge the quality of a study) (Bai et al., 2012; Crowe & Sheppard, 2011; Heyvaert et al., 2013; Santiago-Delefosse et al., 2016).”
Cited by
Items 3,801–3,900 of 4,578
Processo de aprendizagem na simulação clínica pediátrica e o papel das tecnologias educativas
Oliveira, J. F. S., & Struchiner, M. (2026) · Journal of Health Informatics · doi:10.59681/2175-4411.v18.2026.1512
Sijali, K. K., Poudel, T., & Dahal, N. (2026) · Cogent Education · doi:10.1080/2331186x.2026.2633469
Amu, H., Dzadza, A. E. L. A., Glover, R. M., et al. (2026) · BMJ Open · doi:10.1136/bmjopen-2025-110779
General practitioner training in youth suicide prevention: a systematic review
Burke, V., Hudson, C., & Nearchou, F. (2026) · Psychology Health & Medicine · doi:10.1080/13548506.2026.2645954
Laughter-inducing Interventions for Workplace Stress Management: A Scoping Review
Matteau, L., Gilbert‐Ouimet, M., Stefan, T., Demers, J., Jenny, J. I. N., Toupin, I., Medrano, M. C., & LeBlanc, A. (2026) · Safety and Health at Work · doi:10.1016/j.shaw.2026.03.006
Edwards, D. J., Clark, E., Csontos, J. K., Dobbins, M., Gillen, E., Hounsome, J. C., Neil-Sztramko, S. E., Lewis, R., Mann, M. K., & Prue, G. (2026) · Research Square · doi:10.21203/rs.3.rs-8724550/v2
Helm, R. K., Greenhalgh, T., Williamson, J., & Trofimov, A. (2026) · Journal of Criminal Justice · doi:10.1016/j.jcrimjus.2026.102634 · Cited by 2
Hegarty, J. M., Cotter, L., Koç, İ., Brosnan, M. C., Clarke, V., Cliffe, K., Crowley, P., Harnett, G., Aoife, L., Leahy‐Warren, P., et al. (2026) · BMJ Open · doi:10.1136/bmjopen-2025-106351 · Cited by 1
Arriaga, E. B., & Manssur, F. A. G. (2026) · Revista Científica Multidisciplinar G-nerando · doi:10.66473/rcmg.v7i1.1018
Schimmelpfeng, P. H. C., de ARAÚJO, G. C. C., Parente, R. D. C. P., & Brito, R. D. O. (2026) · Educação & Formação · doi:10.25053/redufor.v11.e16184 · Cited by 1
Karachalios, I., Tantaroudas, N. D., Steinhauser, J., et al. (2026) · Open Research Europe · doi:10.12688/openreseurope.21985.1
López-Arenas, D., Sanduvete‐Chaves, S., Mena-Raposo, J., & Moscoso, S. C. (2026) · The Spanish Journal of Psychology · doi:10.1017/sjp.2026.10021 · Cited by 1
“For the comparison with the MQSOM, three instruments were selected: (a) the Methodological Rigor in Mixed Methods (MRMM) checklist (Harrison et al., 2020), the GREOM (Portell et al., 2015), and the Mixed Methods Appraisal Tool (MMAT) checklist (Hong et al., 2018).”
Jones, J., Williams, S., & Davies, S. (2026) · British Journal of Community Nursing · doi:10.12968/bjcn.2025.0238
Nichol, B., Ray, D., Tanner, L., Oliver, E. J., Vlaev, I., Sniehotta, F., & McGowan, L. J. (2026) · Globalization and Health · doi:10.1186/s12992-026-01207-x · Cited by 3
Hooper, M., Joseph, J., Davis, D. L., Blackburn, J., Al-Ghareeb, A. Z., Hampton, K., Atchan, M., Knight-Agarwal, C. R., Frank, G. D., Northam, H. L., et al. (2026) · Vaccine · doi:10.1016/j.vaccine.2026.128528 · Cited by 1
Kongpet, J., Egli, V., Smith, M., & Honey, M. L. L. (2026) · Nursing Praxis in Aotearoa New Zealand · doi:10.36951/001c.158257
Keshtkarhesamabadi, B., Franco, D., Bonito, F. P., Coelho-e-Silva, M. J., & Fragoso, I. J. (2026) · Journal of Voice · doi:10.1016/j.jvoice.2026.01.011
Braun, A., Fringer, A., Gattinger, H., Eissler, C., Shaha, M., Beer, T., & Roes, M. (2026) · JMIR Research Protocols · doi:10.2196/85773
“To support methodological coherence and quality across both strands, the Mixed Methods Appraisal Tool [69] and the GRAMMS guidelines [47,52] are used throughout the design, analysis, and integration phases.”
Turcotte, A.-S., Szulewski, A., McKeown, S., Hajjaj, O., Cowls, M. H., Morrison, C., Zhu, N., Pentland, V., Baril, L., Garceau-Tremblay, Z., et al. (2026) · Canadian Journal of Emergency Medicine · doi:10.1007/s43678-026-01150-1
“Full-text empirical studies were methodologically appraised by 2 independent reviewers using the Mixed Methods Appraisal Tool (MMAT, version 2018) with disagreements resolved through discussion [13, 14].”
Intentions of allied health students to work with older adults: A systematic review
O'Shea, M., & Bartrim, K. (2026) · Educational Gerontology · doi:10.1080/03601277.2026.2649352 · Cited by 1
Hirumi, A. “. C., Giacumo, L. A., Kurzweil, D., Velasco, E. D. L. M., & Moon, H. (2026) · TechTrends · doi:10.1007/s11528-026-01177-2
“…(MERSQI) (Reed et al., 2007), the Critical Appraisal Skills Programme (CASP) Checklists (Singh, 2013), and the Mixed Methods Appraisal Tool (MMAT) (Hong et al., 2018), the R-A-G scale provides a rapid, visual method that enables instructional designers to prioritize evidence while maintaining…”
The role of microfinance in overcoming youth unemployment: a systematic literature review
Magali, J. (2026) · International Journal of Adolescence and Youth · doi:10.1080/02673843.2026.2653160
Hlongwa, M., Tsai, A. C., & Choko, A. T. (2026) · BMJ Open · doi:10.1136/bmjopen-2025-107440
Teacher communities supporting arts education: A systematic literature review
Bhreithiúnaigh, R. N. A., & Kenny, A. (2026) · Teaching and Teacher Education · doi:10.1016/j.tate.2026.105509 · Cited by 1
Guéchi, Y., Noiré, Y., Castelain, T., D’Agostino, R., Ribordy, V., & Féral‐Pierssens, A. (2026) · BMC Emergency Medicine · doi:10.1186/s12873-026-01562-3
“The reporting of our qualitative component adheres to the Consolidated Criteria for Reporting Qualitative Studies (COREQ) checklist (Appendix A) [18] and the methodological quality of the overall mixed design was assessed using the Mixed Methods Appraisal Tool (Appendix B) [19].”
Black African men’s perception of mental health in Africa: Scoping review protocol
Chavalala, L., & Makhado, L. (2026) · PLOS mental health. · doi:10.1371/journal.pmen.0000586
Stephanos, S., & Donaldson, S. I. (2026) · American Journal of Evaluation · doi:10.1177/10982140261438466
“We calculated a composite mixed-methods score as the average of the three component scores (equal weights) and then applied a cap based on the MMAT principle that overall quality cannot exceed the weakest strand (Hong et al., 2018).”
Sofianos, K. C., Kaponis, A., Stefanidakis, M., Μaragoudakis, M., Pappas, T., & Bukauskas, L. (2026) · Engineering Technology & Applied Science Research · doi:10.48084/etasr.13756 · Cited by 1
“Table I presents the assessment dimensions and corresponding scoring criteria applied to the selected studies, based on adaptations of the Newcastle-Ottawa Scale [27, 28].”
Luk, B. H.-K., Polly, H. X., CHUNG, R. W. M., ZHANG, W. W., & Lam, W. (2026) · JMIR Diabetes · doi:10.2196/86767
“The methodological quality of each included study was appraised independently by 2 reviewers using the Mixed Methods Appraisal Tool (MMAT; version 2018; McGill University) [27].”
He, L., & Chen, Z. (2026) · European Journal of Education · doi:10.1111/ejed.70569 · Cited by 1
Martin, A., & Preston, L. (2026) · The Clinical Teacher · doi:10.1111/tct.70367
The implementation of certification procedures at the activity level in French hospitals
Salma, I. (2022) · theses.fr (ABES) · doi:10.70675/38efbd34zfbdbz4a4czabfazaf8c44ee2b82
Biodiesel policy and crude oil price fluctuations in agri commodity markets: A systematic review
Hassan, N. A. M., Khalid, N., Zakaria, K., Kamil, N. N., Ahmad, S. M., SENAWI, R., MAZLAN, R., Salleh, K. M., Abdullah, N., & Ismail, A. (2026) · Multidisciplinary Reviews · doi:10.31893/multirev.2026493
“The MMAT provides assessment criteria for each research design.”
InfluentialMa, Y., Chan, S. K.-Y., Lee, N. T. H., Tse, D. M.-S., Wong, E. S. C., Chu, S. K. W., Holroyd, E. A., & Wong, W. C. W. (2026) · Review of Educational Research · doi:10.3102/00346543261426583
Essex, R., Beri, D., & Jagnoor, J. (2026) · BMC Public Health · doi:10.1186/s12889-026-27234-2 · Cited by 1
“Given the nature of the studies found in the above search, we used the Mixed-Methods Appraisal Tool (MMAT) [14].”
Academic performance of enrolled nurses in Bachelor of Nursing transition programs: A scoping review
Park, S. (. A., Hall, K., Korbay, K., et al. (2026) · Nursing Outlook · doi:10.1016/j.outlook.2026.102758
Naffi, N., Bahlouli, Y. E., Fortier, N., Doumbouya, M. B., Shakeraneh, S., Faraj, D. A., Grégoire, J., Beaulieu, N., & Whelan, K. (2026) · Médiations et médiatisations · doi:10.52358/mm.vi25.507
Akinlotan, O. O., Vidanage, D., Ajibade, B. O., Asamoah, E. N., Faleti, D. D., & Udushirinwa, C. C. (2026) · Journal of Mental Health · doi:10.1080/09638237.2026.2646295
Smith, M. G., Pope, N., Smith, N. L., Henry, G., Honan, I., Corso, K., Evans, G., Doyle, C., & Harvey, A. R. (2026) · Child Care Health and Development · doi:10.1111/cch.70271 · Cited by 1
Rosli, R. M. K. R., Zakaria, A. F., & Amodu, O. A. (2026) · European Journal of Engineering Education · doi:10.1080/03043797.2026.2648688 · Cited by 1
Yeo, E., & Do, S. (2026) · The Journal of Korean Institute of Information Technology · doi:10.14801/jkiit.2026.24.3.169
Dhar, S. R., Alam, M. S., Rony, M. K. K., Rashid, M. M., Shahi, D., Shahidullah, M., Sizan, A. A., Das, S., & Semi, M. M. A. (2026) · International Journal of Developmental Disabilities · doi:10.1080/20473869.2026.2654176 · Cited by 1
Sirparanta, A. (2024) · theses.fr (ABES) · doi:10.70675/01e34829zd16az44abz83d6z94398e631330
Global Service-Learning in Counselor Education: A Systematic Review
Mitchell, K. V., Willhite, M., & Donald, S. (2026) · International Journal for the Advancement of Counselling · doi:10.1007/s10447-026-09644-4
“The MMAT was used to deter - mine the quality of the 15 full-text articles that met the study inclusion criteria.”
InfluentialBraun, A., Fringer, A., Gattinger, H., et al. (2026) · Zürcher Hochschule für Angewandte Wissenschaften digital collection (Zurich University of Applied Sciences) · doi:10.21256/zhaw-36333
The Unyielding Burden: Epidemiological Trends of Foot-and-Mouth Disease in Africa over 100 YearsPreprint
Sopeju, A. E., Fadele, J. S., & Peters, A. R. (2026) · doi:10.12688/verixiv.3073.1 · Cited by 1
Park, J., Cho, H., Park, W. S., & Kim, H. (2026) · International Journal of Older People Nursing · doi:10.1111/opn.70078
“Two authors (JSP and HC) independently assessed the quality of the 21 included studies using the mixed methods appraisal tool (Hong et al. 2018).”
Environmental NGO Advocacy and Governance Transformation in Mining Regions: A Systematic Review
Syam, R., Wisadirana, D., Susilo, E., & Nurhadi, I. (2026) · Environmental Quality Management · doi:10.1002/tqem.70352 · Cited by 1
Jeminiwa, R. N., Yu, J. M. M., Foy, M., Kwon, S., Isaacs, L., Lockstein, D., & Goodstein, D. (2026) · Journal of Addiction Medicine · doi:10.1097/adm.0000000000001694
Aqtam, I. F., Shouli, M. M., & Alkorom, S. M. M. (2026) · Journal of Nursing Management · doi:10.1155/jonm/2678581 · Cited by 1
“The methodological quality of included empirical studies was assessed independently by two reviewers (S.A. and I.A.) using the Mixed Methods Appraisal Tool (MMAT) [20] .”
Gebre, N. T., Afolabi, O. A., Gafer, N., Nkhoma, K. B., Ayers, N., Hanlon, C., & Harding, R. (2026) · JMIR mhealth and uhealth · doi:10.2196/80321 · Cited by 1
“The Mixed Methods Appraisal Tool (MMAT) [39] was used to assess the methodological quality and rigor of the included studies, as it is useful to assess the quality of qualitative, quantitative, and mixed methods studies.”
Rane, E., Bloom, J., Chan, X. W., et al. (2026) · Pediatric Surgery International · doi:10.1007/s00383-026-06424-4
“However, articles were not excluded based on quality appraisal alone, as calculating an overall score is discouraged [13].”
Parida, J., & Das, A. (2026) · Children and Youth Services Review · doi:10.1016/j.childyouth.2026.108976
Mbonda, A., Fai, K. N., Mboungou, T. K., Lemonge, N., Akem, T. E., Ohize, S. O., Hajjar, J. M., Mbarga, N. F., & Kamgno, J. (2026) · BMJ Public Health · doi:10.1136/bmjph-2025-003743 · Cited by 1
“Mixed-methods studies will be evaluated using the Mixed Methods Appraisal Tool 30 or the relevant JBI format.”
Advanced Practice Nursing (APN) in the world: a comparative analysis of selected countries
Żuratyński, P., & Taki, M. (2026) · Pielegniarstwo XXI wieku / Nursing in the 21st Century · doi:10.12923/pielxxiw-2026-0010
Bewket, B., & Pimenta, A. M. (2026) · Frontiers in Clinical Diabetes and Healthcare · doi:10.3389/fcdhc.2026.1794920 · Cited by 1
“The Mixed Methods Appraisal Tool (MMAT, 2018 version) (54), which enables the simultaneous evaluation of qualitative, quantitative, and mixed-methods research within a uni fi ed framework, was used to evaluate the methodological quality of all included studies.”
Botha, C., Jacobs, A., Roux, L. L., Sihlobo, Y., & Syster, C. (2026) · Commonwealth Youth and Development · doi:10.25159/2663-6549/19604
Addressing return to driving goals in the community after acquired brain injury: a scoping review
James, S., Gustafsson, L. A., Watter, K., Nielsen, M., & Bassingthwaighte, L. (2026) · Disability and Rehabilitation · doi:10.1080/09638288.2026.2655110
Mjuza, L., Maimela, C., Nekhumbe, A., Ramatsokotla, S., Duah, E., & Mashamba-Thompson, T. P. (2026) · Advances in Public Health · doi:10.1155/adph/9824199
Bowden, J., Frude, A., & Senior, J. (2026) · Journal of Forensic Psychiatry and Psychology · doi:10.1080/14789949.2026.2653637
Phua, S. S. L., Lau, Y., Ang, W. W., Nur, S. B. H., Pang, P. C., & Wong, S. H. (2026) · Anxiety Stress & Coping · doi:10.1080/10615806.2026.2659812
Dell’Anna, S., Frizzarin, A., & Biagiotti, G. (2026) · Teaching and Teacher Education · doi:10.1016/j.tate.2026.105559
Kavak, E., Örsal, Ö., & Duru, P. (2026) · STED / Sürekli Tıp Eğitimi Dergisi · doi:10.17942/sted.1529135
Haq, N. U., Yin, Y., Mehmood, I., & Fareed, M. (2026) · Corporate Social Responsibility and Environmental Management · doi:10.1002/csr.70607 · Cited by 3
Baulckim, L. T. (2026) · INQUIRY The Journal of Health Care Organization Provision and Financing · doi:10.1177/00469580261441133
“45”
Generative AI and adaptive systems for customising help-seeking scaffolds: A systematic review
Jecha, J., Igweani, C. C., Banaras, A., Liang, Y., & Mwalyego, F. M. (2026) · Information Services & Use · doi:10.1177/18758789261486037
Ren, C., Bailey, R. P., Samsudin, N., Mishra, P. K., Yin, S., & Cui, J. (2026) · Cogent Education · doi:10.1080/2331186x.2026.2655942 · Cited by 1
King, D. D., Cadieux, C. S., & Beason, F. A. (2026) · Research in Gerontological Nursing · doi:10.3928/19404921-20260416-01
Alsararatee, H. H., Dairo, Y. M., Burton, H., Whitby, S., & Thorburn, M. (2026) · International Journal for Advancing Practice · doi:10.12968/ijap.2026.0008
SYNTHETIC INTELLIGENCE APPLIED TO RETAIL MARKETING
Arándiga, S. (2026) · Revista Gênero e Interdisciplinaridade · doi:10.51249/gei.v7i02.2953
Katsaros, K., Traue, A., & Huang-Lachmann, J.-T. (2026) · BMC Public Health · doi:10.1186/s12889-026-27435-9 · Cited by 1
“Due to many different study designs included, adapted Critical Appraisal Skills Program (CASP) checklists were selected corresponding to each design type; the Mixed Method Appraisal Tool (MMAT) was used for mixed-method approaches [35, 36].”
Schneider, J., Smith, H. G., Paraskeva, N., & Craddock, N. (2026) · Body Image · doi:10.1016/j.bodyim.2026.102093
Bozkurt, A., Ergezen, F. D., Koren, Y., & Hayman, L. L. (2026) · Global Nursing Insights · doi:10.31125/globnursinsights.1753972
Occupational safety in coal logistics: a systematic review of systemic risks and human factors
Abdullah, W. P. A.-A. W., Arifin, K., Juhari, M. L., et al. (2026) · Frontiers in Public Health · doi:10.3389/fpubh.2026.1824250
“Given the methodological heterogeneity of the included studies, which comprise empirical research, simulations, and case studies, the Mixed Methods Appraisal Tool (MMAT) version 2018 was employed (25).”
Mobile Learning and Competency-Based Education in TVET: A Bibliometric Synthesis
Wagino, W., Tymofiiv, V., Polin, K., Hidayat, N., Muslim, M. H. P., & Koto, R. D. (2026) · International Journal of Interactive Mobile Technologies (iJIM) · doi:10.3991/ijim.v20i08.61205
Addo, S., Dabi, M., Anyan, K. F., Samuel, D., Duodu, C. P., & Diyie, R. L. (2026) · Journal of the World Aquaculture Society · doi:10.1111/jwas.70093 · Cited by 1
Razak, R. S. A., Amsah, N., Heng, P. P., Ling, M. Y. J., Mohtar, N., Masdor, N. A., Othman, R., Hod, R., & Idris, I. B. (2026) · BMC Public Health · doi:10.1186/s12889-026-27501-2
“…using the Mixed-Methods Appraisal Tool (MMAT), a critical appraisal tool designed to appraise the quality of the empirical, non-experimental studies from five categories i.e. randomized controlled trials, non-randomized studies, quantitative descriptive studies, and mixed-methods studies [26].”
Home Healthcare in South Korea: A Literature Review on Access, Quality, and Cost
Kim, D. H., Jones, D. A., Guerrazzi-Young, C., & Amin, R. (2026) · International Journal of Health Policy and Management · doi:10.34172/ijhpm.9207
“22,23 The Mixed Methods Appraisal Tool allows the use of a single instrument for the assessment of qualitative, quantitative randomized controlled trials, quantitative non-randomized, quantitative, and mixed methods studies.”
Bahamondes-Álvarez, R., van Hulst, B. M., Lund, C., Nooteboom, L. A., Oehm, L. M., Woermann, E., & Vermeiren, R. R. J. M. (2026) · Community Mental Health Journal · doi:10.1007/s10597-026-01638-1 · Cited by 1
Ndacayisaba, G., Vanderweyen, J., Mapinduzi, J., Nduwimana, I., Sinzakaraye, A., Deboeck, G., Bonnechère, B., & CANT, J. V. (2026) · Physiotherapy Theory and Practice · doi:10.1080/09593985.2026.2664643 · Cited by 1
Fatai, A., Sattayarom, C., & Butdee, S. (2026) · Western Journal of Nursing Research · doi:10.1177/01939459261442054
Measurement Matters: A Meta-Analytic Examination of Transactive Memory Systems and Team Outcomes
Fausett, C. M., Keebler, J. R., Lazzara, E. H., Gregory, M., & Blickensderfer, B. (2026) · Small Group Research · doi:10.1177/10464964261434540
Elgazzar, S. H., Ibrahim, R. K., Ayoub, Y. M., Alanazi, R., Alanazi, A. K., Tantawi, H. R., Al-Qudimat, A. R., Alsharidah, A., & Hendy, A. (2026) · Frontiers in Medicine · doi:10.3389/fmed.2026.1718431
Relationship Between Reading and Working Memory in Children and Adolescents: a Meta-Analysis
Looi, L. L., Lim, W. P. H., Wu, C., O'BRIEN, B. A., Yeo, D. J. S., & Chen, S. A. (2026) · Educational Psychology Review · doi:10.1007/s10648-026-10151-6
Amoah-Oppong, D., & ACHEAMPONG, B. (2026) · Research Square · doi:10.21203/rs.3.rs-9544668/v1
A Literature Review on Vietnamese Teachers’ Integration of AI-Based Tools in Lesson Planning
Nguyễn, N. H., & Ho, H. H. (2026) · International journal of AI in language education. · doi:10.54855/ijaile.26314
Liu, B. (2026) · Support for Learning · doi:10.1111/1467-9604.70052
Kusuma, N. H. S., Irnandi, D. F., Pakpahan, C., et al. (2026) · The Journal of Sexual Medicine · doi:10.1093/jsxmed/qdag121
Coaching programmes in undergraduate medical education: An integrative review protocol
Weber, L., Freeman, J., Gysin, S., Florin, J., Sondén, A., Schaer, D. J., & Guttormsen, S. (2026) · F1000Research · doi:10.12688/f1000research.179726.1
“The methodological quality of all eligible empirical studies will be appraised using the Mixed Methods Appraisal Tool, 15 which allows for the assessment of qualitative, quantitative, and mixed-methods studies within a single framework.”
Opening new doors: a realist review on return to work with a new employer for sick-listed employees
Klerkx, M.-L., van Hees, S., Hutting, N., de Rijk, A. E., & Oomens, S. P. C. J. (2026) · Disability and Rehabilitation · doi:10.1080/09638288.2026.2660004
Abdullah, C. U., Rahmawati, S., Cotton, W., & Peralta, L. R. (2026) · Education Sciences · doi:10.3390/educsci16050725
Silva, B. L., da Silva, L. S. C. V., da Silva, W. V., Oliveira, V. J. S., Pereira, L. D. A., Silva, R. D. F., Su, Z., & da Veiga, C. P. (2026) · Families in Society The Journal of Contemporary Social Services · doi:10.1177/10443894261417718
Psychological flexibility-based interventions for teacher well-being: A systematic review
Kämper, E., Lampinen, V. S., Katajavuori, N., Carvalho, T. D. S., & Asikainen, H. (2026) · Teaching and Teacher Education · doi:10.1016/j.tate.2026.105591 · Cited by 2
The Use of Digital Storytelling in Early Childhood Education: A Systematic Review
Sakır, A. (2026) · MIER Journal of Educational Studies Trends & Practices · doi:10.52634/mier/2026/v16/i1/2899 · Cited by 3
“Future studies should employ formal sensitivity analyses, such as synthesising studies with higher MMAT scores, to verify the stability of conclusions.”
InfluentialFitzgerald, A., James, P. D., Hollywood, E., Comiskey, C. M., Reilly, R., Annunziata, K. N., & Kelly, P. (2026) · Global Implementation Research and Applications · doi:10.1007/s43477-026-00235-4 · Cited by 1
Rajkumar, E., Rahul, P., Devi, R. R., Gopi, A., Shivi, & Romate, J. (2026) · Journal of Technology in Behavioral Science · doi:10.1007/s41347-026-00639-x
A Systematic Review of Research on Communication in Traditional Chinese Medicine
Chen, L., Chinn, D., & Tang, C. (2026) · Health Communication · doi:10.1080/10410236.2026.2655768 · Cited by 1
Sholikhah, A. N. H., Kaloeti, D. V. S., Bhima, S. K. L., & Indrawati, N. D. (2026) · Youth Justice · doi:10.1177/14732254261428270
Chen, X. S., Wen, W., Choi, S., Andrade, F. C. D., & Mejía, S. T. (2026) · Aging & Mental Health · doi:10.1080/13607863.2026.2664078 · Cited by 2
Davies, R. L., Clark, J. C., Martin, M., & Rice, K. (2026) · Frontiers in Psychology · doi:10.3389/fpsyg.2026.1820872
“Currently, multiple tools support critical appraisal of research, such as [Joanna Briggs Institute (JBI), 2020], Mixed Methods Appraisal Tool (Hong et al., 2018), and the Grading of Recommendations Assessment, Development and Evaluation (GRADE) tool which provides a structured approach to evaluating and rating the cer…”
Metadata
- Authors
- Hong, Q. N., Fàbregues, S., Bartlett, G., Boardman, F., Cargo, M., Dagenais, P., Gagnon, M.-P., Griffiths, F., Nicolau, B., O’Cathain, A., Rousseau, M.-C., Vedel, I., & Pluye, P.
- Affiliations
- McGill University (CA)
- Universitat Oberta de Catalunya (ES)
- University of Warwick (GB)
- University of Canberra (AU)
- Université de Sherbrooke (CA)
- Université Laval (CA)
- University of Sheffield (GB)
- Institut National de la Recherche Scientifique (CA)
- Corresponding author
- Pluye, Pierre
- ORCID
- Venue
- Education for Information
- Publisher
- SAGE Publications
- Published
- 2018-12-18
- Volume
- 34
- Issue
- 4
- Pages
- 285-291
- ISSN
- 0167-8329 (ISSN-L)
- 1875-8649
- OpenAlex
- W2901724277
- Cited by
- 4,737
- References
- 21
- Citations by year
- 2026: 1312 · 2025: 974 · 2024: 800 · 2023: 610 · 2022: 501 · 2021: 370 · 2020: 143 · 2019: 22 · 2018: 3
- FWCI
- 102.21
- Type
- journal article
- Language
- en
- Open access
- green
- hdl.handle.net
- Licence
- cc-by-nc-nd
- Locations
- Education for Information (publishedVersion)
- O2 - Repositori Institucional (Universitat Oberta de Catalunya) (publishedVersion, cc-by-nc-nd)
- RECERCAT (Consorci de Serveis Universitaris de Catalunya) (publishedVersion, cc-by-nc-nd)
- White Rose Research Online (University of Leeds, The University of Sheffield, University of York) (acceptedVersion)
- Warwick Research Archive Portal (University of Warwick) (acceptedVersion)
- HAL (submittedVersion)
- eScholarship@McGill (McGill) (submittedVersion)
- LA Referencia (Red Federada de Repositorios Institucionales de Publicaciones Científicas) (submittedVersion, cc-by-nc-nd)
- eScholarship@McGill (McGill) (submittedVersion, cc-by-nc-nd)
- EspaceINRS (National Institute for Scientific Research (Canada)) (submittedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Sociology and Political Science › Delphi Technique in Research
- Social Sciences › Education › Qualitative Research Methods and Applications
- Keywords
- Mixed Methods Appraisal Tool, critical appraisal, study quality assessment, mixed methods, systematic reviews, qualitative research, quantitative research, randomized controlled trials, non-randomized studies, e-Delphi, inter-rater reliability
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.