The Mixed Methods Appraisal Tool (MMAT) version 2018 for information professionals and researchers
Open accessThe DOI matches its Crossref record; 21 of the 21 references it cites were found in a registry or an index; 2 of the 26 sources its claims cite were checked for support.
Abstract
INTRODUCTION: Appraising the quality of studies included in systematic reviews combining qualitative and quantitative evidence is challenging. To address this challenge, a critical appraisal tool was developed: the Mixed Methods Appraisal Tool (MMAT). The aim of this paper is to present the enhance ments made to the MMAT. DEVELOPMENT: The MMAT was initially developed in 2006 based on a literature review on systematic reviews combining qualitative and quantitative evidence. It was subject to pilot and interrater reliability testing. A revised version of the MMAT was developed in 2018 based on the results from usefulness testing, a literature review on critical appraisal tools and a modified e-Delphi study with methodological experts to identify core criteria. TOOL DESCRIPTION: The MMAT assesses the quality of qualitative, quantitative, and mixed methods studies. It focuses on methodological criteria and includes five core quality criteria for each of the following five categories of study designs: (a) qualitative, (b) randomized controlled, (c) nonrandomized, (d) quantitative descriptive, and (e) mixed methods. CONCLUSION: The MMAT is a unique tool that can be used to appraise the quality of different study designs. Also, by limiting to core criteria, the MMAT can provide a more efficient appraisal.
Overview
CoChat Summary
TL;DR
This paper presents version 2018 of the Mixed Methods Appraisal Tool (MMAT), a single critical appraisal tool for assessing the methodological quality of qualitative, quantitative, and mixed methods studies in mixed studies reviews. It describes the tool's 25 criteria across five study design categories, its development history, and guidance for use.
Key findings
The MMAT version 2018 includes a total of 25 criteria and 2 screening questions, covering five categories of study designs: (a) qualitative, (b) randomized controlled trial, (c) nonrandomized, (d) quantitative descriptive, and (e) mixed methods, with five core criteria per category rated as yes, no, or can't tell. The tool was developed from a 2006 first version (15 criteria across four categories), revised in 2011, and further revised into the 2018 version based on usefulness testing with 20 researchers, a literature review on critical appraisal tools, and a modified e-Delphi study with 73 methodological experts. Two interrater reliability studies used 32 and 261 papers respectively and showed the need to clarify criteria, particularly for nonrandomized and qualitative studies. The paper argues the MMAT offers a time-efficient alternative by limiting to core criteria and uniquely includes specific criteria for mixed methods studies, though its criteria are harder to judge because they target methodological rather than reporting quality.
Methods
This is a tool description paper (third in a three-part series on systematic reviews for information professionals) rather than an empirical study; it narrates the development of the MMAT and explains how to apply it. Development drew on a literature review on mixed studies reviews, pilot testing in workshops with four reviewers appraising six studies, two interrater reliability studies (32 and 261 papers), usefulness testing via interviews with 20 researchers, a literature review on critical appraisal tools, and a modified e-Delphi study with 73 methodological experts. Use involves three steps: two optional screening questions, selection of the appropriate design category (or categories), and rating each criterion as yes, no, or can't tell, with detailed presentation of ratings advised over an overall numerical score.
Limitations
The paper states that revision of the MMAT has so far focused on content validity and usefulness, and that further testing on its validity and reliability will be needed; it also notes that modifications may be necessary as evidence develops. It states that MMAT criteria are more difficult to judge than other appraisal tools because they focus on methodological quality rather than reporting quality. Evident from the text: the tool's development evidence base is described only in summary form, with reliability and validity results reported by reference to prior studies rather than presented here.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
2 Valid · 0 Not valid · 18 Not checked · 11 Own results and interpretations
Every source the paper cites for this claim was checked and supports it.
In the paper
“To further its development, usefulness testing was performed by interviewing 20 researchers who have used the MMAT in a systematic review or have contacted the developer of the tool for questions or permission to use the tool (18).”
What the sources say
- Supportedcites [18] 18. Hong QN, Gonzalez-Reyes A, Pluye P. Improving the usefulness of a tool for appraising the quality of qualitative, quantitative and mixed methods studies, the Mixed Methods Appraisal Tool (MMAT). J Eval Clin Pract. 2018;24(3):459-67. source
The Abstract reports semi-structured interviews with 20 participants to explore the MMAT’s usefulness. The Study Participants section says researchers were recruited through systematic reviews that used the MMAT and a list of people who had contacted its developer for permission or guidance. (LLM, conf=0.94)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“The results of this study were helpful to identify changes to be made in the MMAT.”
What the sources say
- Supportedcites [18] 18. Hong QN, Gonzalez-Reyes A, Pluye P. Improving the usefulness of a tool for appraising the quality of qualitative, quantitative and mixed methods studies, the Mixed Methods Appraisal Tool (MMAT). J Eval Clin Pract. 2018;24(3):459-67. source
The Discussion says the study explored MMAT users’ experiences to help revise the tool and identified themes suggesting areas for improvement. The Recommendations for Improving the MMAT then proposes specific changes on the basis of those results. (LLM, conf=0.94)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
In the paper
“Moreover, it includes specific criteria for mixed methods studies, which is not often found in other tools (21).”
What the sources say
- Not matchedcites [21] 21. Pluye P. Critical appraisal tools for assessing the methodological quality of qualitative, quantitative and mixed methods studies included in systematic mixed studies reviews. J Eval Clin Pract. 2013;19(4):722.
We searched by its title and found no matching paper.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Not matched
In the paper
“To address this challenge, a critical appraisal tool for assessing the quality of quantitative, qualitative and mixed methods studies was developed: the Mixed Methods Appraisal Tool (MMAT) (7).”
What the sources say
- Not checked — no full text of the sourcecites [7] 7. Pluye P, Gagnon MP, Griffiths F, Johnson-Lafleur J. A scoring system for appraising mixed methods research, and concomitantly appraising qualitative, quantitative and mixed methods primary studies in mixed studies reviews. Int J Nurs Stud. 2009;46(4):529-46. source
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“Moreover, it includes specific criteria for mixed methods studies, which is not often found in other tools (21).”
What the sources say
- Not matchedcites [21] 21. Pluye P. Critical appraisal tools for assessing the methodological quality of qualitative, quantitative and mixed methods studies included in systematic mixed studies reviews. J Eval Clin Pract. 2013;19(4):722.
We searched by its title and found no matching paper.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Not matched
5 more claims
Create a free account to see all 31 claims and to verify the 1 reference still open.
- It may state two separate claims as one. 7
- Probably not a substantial claim. 16
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Quality was assessed using two validated tools by study design. Quantitative studies were appraised using the Mixed Methods Appraisal Tool Version 2018, which evaluates five methodological criteria on a binary scale (Hong et al., 2018). Qualitative studies were additionally appraised using the Critical Appraisal Skills Programme Qualitative Checklist, which evaluates 10 criteria (CASP UK, 2018).”
From Cultural stigma and caregiver burden among family caregivers of people with nervous system disorders in the Middle East and North Africa: A mixed-methods systematic review (2026) · cites this paper as Hong et al., 2018
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“The quality of the included studies was assessed using the 2018 version of the Mixed Methods Appraisal Tool (MMAT).15”
From Psychosocial interventions for siblings of children with cancer: A mixed methods systematic review (2021) · cites this paper as [15]
The cited source says what the claim attributes to it.
In the citing paper
“To further limit risks of bias, a formal risk of bias assessment tool (ROB-tool), the Mixed Methods Appraisal Tool [52], was used to evaluate the overall study quality.”
From Mediators of outcome in adolescent psychotherapy and their implications for theories and mechanisms of change: a systematic review (2024) · cites this paper as [52]
The cited source says what the claim attributes to it.
In the citing paper
“LM, with the support of a research assistant, assessed the methodological quality of empirical studies (qualitative, quantitative and mixed methods) using the Mixed Methods Appraisal Tool (MMAT) [45].”
From Transnationalism and care of migrant families during pregnancy, postpartum and early-childhood: an integrative review (2020) · cites this paper as [45]
The cited source says what the claim attributes to it.
In the citing paper
“The quality of the included studies was independently assessed by two authors using the Mixed Methods Appraisal Tool (MMAT) (Hong et al., 2018).Disagreements were resolved through discussion.”
From Gamification on Mathematics Engagement and Motivation in Secondary School and Higher Education: A Systematic Review and Meta-Analysis (2026) · cites this paper as Hong et al., 2018
29 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ 4. Conclusion · p. 5
“## 4. Conclusion”
§ 4. How to use the MMAT? · p. 4
“## 4. How to use the MMAT?”
Citations
Cites
Pace, R., Pluye, P., Bartlett, G., Macaulay, A. C., Salsberg, J., Jagosh, J., & Seller, R. (2012) · International Journal of Nursing Studies · doi:10.1016/j.ijnurstu.2011.07.002 · Cited by 1,331 · cited 2×
“Then, two interrater reliability studies on the MMAT were conducted using respectively 32 and 261 papers (Pace et al., 2012; Souto et al., 2015).”
Self-citationPluye, P., Gagnon, M.-P., Griffiths, F., & Johnson-Lafleur, J. (2009) · International Journal of Nursing Studies · doi:10.1016/j.ijnurstu.2009.01.009 · Cited by 1,119 · cited 3×
“This led to develop a first version of the MMAT that is in line with a social constructionist worldview (Pluye et al., 2009).”
InfluentialSelf-citationHong, Q. N., Gonzalez‐Reyes, A., & Pluye, P. (2018) · Journal of Evaluation in Clinical Practice · doi:10.1111/jep.12884 · Cited by 1,107 · cited 3×
“To further its development, usefulness testing was performed by interviewing 20 researchers who have used the MMAT in a systematic review or had contacted the developer of the tool for questions or permission to use the tool (Hong et al., 2018a).”
Self-citationSystems to rate the strength of scientific evidence.
West, S. L., King, V., Carey, T. S., Lohr, K. N., Mckoy, N. A., Sutton, S. F., & Lux, L. J. (2002) · PubMed Central · Cited by 923
Zaza, S., Wright-De Agüero, L. K., Briss, P. A., Truman, B. I., Hopkins, D. P., Hennessy, M. H., Sosin, D. M., Anderson, L., Carande-Kulis, V. G., Teutsch, S. M., et al. (2000) · American Journal of Preventive Medicine · doi:10.1016/s0749-3797(99)00122-1 · Cited by 464 · cited 1×
“The algorithm was developed based on several existing algorithms of quantitative study designs (Hartling et al., 2010; Hartling et al., 2011; National Institute for Health Care Excellence, 2012; Scottish Intercollegiate Guidelines Network; Seo et al., 2016; West et al., 2002; Zaza et al., 2000).”
Souto, R. Q., Khanassov, V., Hong, Q. N., Bush, P. L., Vedel, I., & Pluye, P. (2015) · International Journal of Nursing Studies · doi:10.1016/j.ijnurstu.2014.08.010 · Cited by 375 · cited 1×
“Then, two interrater reliability studies on the MMAT were conducted using respectively 32 and 261 papers (Pace et al., 2012; Souto et al., 2015).”
Self-citationA review of critical appraisal tools show they lack rigor: Alternative tool structure is proposed
Crowe, M., & Sheppard, L. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2010.02.008 · Cited by 261 · cited 2×
“Currently, there exist more than 500 critical appraisal tools (i.e., checklists including a list of criteria to judge the quality of a study) (Bai et al., 2012; Crowe & Sheppard, 2011; Heyvaert et al., 2013; Santiago-Delefosse et al., 2016).”
Carroll, C., Booth, A., & Lloyd-Jones, M. (2012) · Qualitative Health Research · doi:10.1177/1049732312452937 · Cited by 248 · cited 1×
“Methodological quality criteria are more difficult to interpret because the reviewers need to judge whether the reported results of a study are trustworthy (Carroll et al., 2012; Hong & Pluye, 2018a).”
Adjustment of meta-analyses on the basis of quality scores should be abandoned
Herbison, P., Hay-Smith, J., & Gillespie, W. J. (2006) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2006.03.008 · Cited by 207 · cited 1×
“Currently in the literature, it is discouraged to calculate an overall score from the ratings of each criterion (Herbison et al., 2006; Higgins & Green, 2008; Viswanathan et al., 2012).”
Critical Appraisal of Mixed Methods Studies
Heyvaert, M., Hannes, K., Maes, B., & Onghena, P. (2013) · Journal of Mixed Methods Research · doi:10.1177/1558689813479449 · Cited by 168 · cited 1×
“Currently, there exist more than 500 critical appraisal tools (i.e., checklists including a list of criteria to judge the quality of a study) (Bai et al., 2012; Crowe & Sheppard, 2011; Heyvaert et al., 2013; Santiago-Delefosse et al., 2016).”
Cited by
Items 801–900 of 4,578
Maun, R., Fabri, M., & Trevorrow, P. (2023) · Journal of Autism and Developmental Disorders · doi:10.1007/s10803-023-06015-5 · Cited by 35
“All 18 papers were quality appraised using the Mixed Meth-ods Appraisal Tool (MMAT) by all three reviewers, primarily < author 1 > and < author 2 > , with < author 3 > reviewing the quality where disagreements occurred.”
MethodInfluentialMoreno, A., Scola, M.-C., Sun, H., Durce, H., Couve, C., Acevedo, K., & GUTMAN, G. M. (2024) · Frontiers in Psychology · doi:10.3389/fpsyg.2023.1237694 · Cited by 43
“Three independent reviewers (HS, KA, M-CS) evaluated the quality of the empirical studies included in the present systematic review with the Mixed Methods Appraisal Tool (MMAT – Hong et al., 2018).”
MethodMcKenna, C., Moyo, B., & Goodwin, J. (2024) · International Journal of Mental Health Nursing · doi:10.1111/inm.13302 · Cited by 27
Rangachari, P., Thapa, A., Sherpa, D. L., Katukuri, K., Ramadyani, K., Jaidi, H. M., & Goodrum, L. (2024) · Frontiers in Public Health · doi:10.3389/fpubh.2024.1413205 · Cited by 23
“As indicated, most initiatives fell under Social and Community Context (44), followed by Neighborhood and Built Environment (40), Economic Stability (39), Healthcare Access and Quality (22), and Education Access and Quality (8).”
BackgroundMethodInfluentialPredictive factors for return to work among cancer survivors in Europe: A systematic review
Schellack, S. K., Breidenbach, C., Rick, O., & Kowalski, C. (2024) · Critical Reviews in Oncology/Hematology · doi:10.1016/j.critrevonc.2024.104422 · Cited by 25
Kılıç, A., Zhou, X., Moon, Z., Hamada, Y., Duong, T., Layton, C., Jhuree, S., Abubakar, I., Rangaka, M. X., & Horne, R. (2025) · BMC Public Health · doi:10.1186/s12889-024-20868-0 · Cited by 23
Mental health professionals’ attitudes towards virtual reality therapies: a systematic review
Bin, S., Alrashdi, D. H., Whitehead, T., Riches, S., & Drini, E. (2025) · Journal of Technology in Behavioral Science · doi:10.1007/s41347-025-00499-x · Cited by 21
“The quality of the included studies was assessed using the Mixed Methods Appraisal Tool (MMAT; Hong et al., 2018) and the Critical Appraisal Skills Programme Qualitative Studies Checklist (CASP) (Critical Appraisal Skills Programme, 2018).”
Maier, M., Powell, D., Murchie, P., & Allan, J. (2025) · Health Psychology Review · doi:10.1080/17437199.2025.2513916 · Cited by 34
“The quality of qualitative and quantitative studies was assessed using the Mixed Methods Appraisal Tool (Hong et al., 2018), opinion, narrative overview, and editorial articles were assessed using the JBI Critical Appraisal Checklist for Text and Opinion Papers ( JBI Manual for Evidence Synthesis ,…”
InfluentialEgwudo, A. E., Jegede, A. O., Oyeniyi, T. A., Ezekwelu, N. J., Abdu-Aguye, S. N., Okwuraiwe, A. P., Onyeaghala, C. A., Ozoude, T. O., Suleiman, M. O., Aziken, G. O., et al. (2025) · PLOS Digital Health · doi:10.1371/journal.pdig.0000928 · Cited by 52
“The methodological quality of the included studies was appraised using the Mixed Method Appraisal Tool [25].”
Crossley, K. M., Whittaker, J. L., Patterson, B., Shill, I., Heming, E. E., Bullock, G. S., Dijkstra, H. P., Donaldson, A., McKay, C. D., Mountjoy, M. L., et al. (2025) · British Journal of Sports Medicine · doi:10.1136/bjsports-2025-110889 · Cited by 37
Supporting self-care of long-term conditions in people with dementia: A systematic review
Rees, J., Tuijt, R., Burton, A., Walters, K., & Cooper, C. (2021) · International Journal of Nursing Studies · doi:10.1016/j.ijnurstu.2019.103432 · Cited by 21
Thodé, M., Pasman, H. R. W., van Vliet, L. M., Damman, O. C., Ket, J. C. F., Francke, A. L., & Jongerden, I. P. (2020) · BMJ Supportive & Palliative Care · doi:10.1136/bmjspcare-2020-002284 · Cited by 23
Khalifeh, A. H., & Hamdan‐Mansour, A. M. (2020) · Journal of Psychosocial Nursing and Mental Health Services · doi:10.3928/02793695-20201015-05 · Cited by 31
Norton, J., & Southon, N. (2020) · Physical & Occupational Therapy In Pediatrics · doi:10.1080/01942638.2020.1836705 · Cited by 30
“As recommended in these guidelines, a detailed presentation of the ratings of each criterion has been presented to inform the quality of included studies (Hong et al., 2018) (Table 2).”
BackgroundMethodInfluentialKhurshid, Z., Brún, A. D., Martin, J. R., & McAuliffe, É. (2021) · Journal of Continuing Education in the Health Professions · doi:10.1097/ceh.0000000000000331 · Cited by 21
“The Mixed Methods Appraisal Tool (MMAT) was used to evaluate methodological quality.(16) The validity and reliability of the MMAT has been established and is suitable for appraising mixed method studies.”
MethodMogharbel, A., Dowding, D., & Ainsworth, J. D. (2021) · JMIR Medical Informatics · doi:10.2196/22923 · Cited by 26
Developing Leadership Talent in Adolescents and Emerging Adults: A Systematic Review
Meyer, M. S., & Rinn, A. N. (2021) · Gifted Child Quarterly · doi:10.1177/00169862211007556 · Cited by 32
Market segmentation in the health tourism industry: a systematic review of approach and criteria
Torkzadeh, L., Jalilian, H., Zolfagharian, M., Torkzadeh, H., Bakhshi, M., & Zarnaq, R. K. (2021) · Journal of Policy Research in Tourism Leisure and Events · doi:10.1080/19407963.2021.1988622 · Cited by 30
van Os, S., Syversen, A., Whitaker, K. L., Quaife, S. L., Janes, S. M., Jallow, M., & Black, G. B. (2021) · Psycho-Oncology · doi:10.1002/pon.5846 · Cited by 21
van der Werf, H. M., Luttik, M. L., de Boer, A., Roodbol, P. F., & Paans, W. (2022) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph19020855 · Cited by 28
“We used the relevant Mixed Methods Appraisal Tool [24] and the Joanna Briggs Institute’s [25] critical appraisal checklists, fitting the different study designs.”
MethodReimagining the role of nursing education in emissions reduction
Ward, A., Heart, D., Richards, C., Bayliss, L. T., Holmes, M. E., Keogh, S. J., & Best, O. (2022) · Teaching and learning in nursing · doi:10.1016/j.teln.2022.02.003 · Cited by 22
A Systematic Review on Navigation Programs for Persons Living with Dementia and Their Caregivers
Kokorelias, K. M., Shiers-Hanley, J. E., Li, Z., & Hitzig, S. L. (2023) · The Gerontologist · doi:10.1093/geront/gnac054 · Cited by 21
Al-Naabi, I., Kelder, J., & Carr, A. R. (2021) · Journal of University Teaching and Learning Practice · doi:10.53761/1.18.5.4 · Cited by 34
Tessarolo, E., Alkhouri, H., Lelos, N., Sarrami, P., & McCarthy, S. M. (2022) · Emergency Medicine Australasia · doi:10.1111/1742-6723.13993 · Cited by 27
Domestic/Family Homicide: A Systematic Review of Empirical Evidence
Truong, M., Yeganeh, L., Cartwright, A. V., Ward, E., Ibrahim, J., Cuschieri, D., Dawson, M., & Bugeja, L. (2022) · Trauma, Violence, & Abuse · doi:10.1177/15248380221082084 · Cited by 24
“For mixed methods studies, we used the “mixed methods studies” section of the Mixed Methods Appraisal Tool version 2018 (MMAT) (Hong et al., 2018).”
MethodInfluentialNurses’ roles in changing practice through implementing best practices: A systematic review
ten Ham‐Baloyi, W. (2022) · Health SA Gesondheid · doi:10.4102/hsag.v27i0.1776 · Cited by 29
“…Epidemiology (STROBE) Statement: guidelines for reporting observational studies (Von Elm et al. 2007) (n = 16), Mixed Methods Appraisal Tool (MMAT) (Hong et al. 2018) (n = 1) and the Effective Practice and Organization of Care Risk of Bias (EPOC RoB) tool for retrospective observational studies…”
MethodInfluentialEvaluating Postvention Services and the Acceptability of Models of Postvention: A Systematic Review
Abbate, L., Chopra, J., Poole, H. M., & Saini, P. (2022) · OMEGA - Journal of Death and Dying · doi:10.1177/00302228221112723 · Cited by 30
“Methodological quality was evaluated by the primary researcher using guidelines stipulated in the Mixed Methods Appraisal Tool (MMAT) (Hong et al., 2018).”
BackgroundMethodInfluentialWhat are the benefits and challenges of mentoring in midwifery? An integrative review
Wissemann, K., Bloxsome, D., Leo, A. D., & Bayes, S. J. (2022) · Women s Health · doi:10.1177/17455057221110141 · Cited by 27
Aworinde, J., Ellis-Smith, C. I., Gillam, J., Roche, M., Coombes, L., Yorganci, E., & Evans, C. (2022) · Alzheimer s & Dementia Translational Research & Clinical Interventions · doi:10.1002/trc2.12304 · Cited by 27
Exploring support strategies for improving nursing student retention
Edge, D., & Gladstone, N. (2022) · Nursing Standard · doi:10.7748/ns.2022.e11914 · Cited by 24
Nothacker, J., Nury, E., Mathieu, M. R., Raatz, H., Meerpohl, J. J., & Schmucker, C. M. (2022) · BMJ Sexual & Reproductive Health · doi:10.1136/bmjsrh-2022-201515 · Cited by 27
Watkins, S., Isichei, O., Gentles, T. L., Brown, R., Percival, T., Sadler, L., Gorinski, R., Crengle, S. M., Cloete, E., de Laat, M. W. M., et al. (2022) · Pediatric Cardiology · doi:10.1007/s00246-022-03006-8 · Cited by 22
Roach, A., Cullinan, S., Shafran, R., Heyman, I., & Bennett, S. (2023) · British Medical Bulletin · doi:10.1093/bmb/ldad001 · Cited by 21
“Quality assessment of included articles was conducted using the Mixed Methods Appraisal Tool.22 This tool was appropriate to use as it allowed for simultaneous evaluation of different study designs.”
MethodHaltaufderheide, J., Lucht, A., Strünck, C., & Vollmann, J. (2023) · Science and Engineering Ethics · doi:10.1007/s11948-022-00419-9 · Cited by 27
Islam, S., Appleton, R., Hutchings-Hay, C., Lloyd‐Evans, B., & Johnson, S. (2023) · PLoS ONE · doi:10.1371/journal.pone.0282157 · Cited by 23
“The MMAT was used to inform us about the quality of information provided about implementation barriers and facilitators.”
MethodInfluentialPsychological well-being of hospice staff: systematic review
Papworth, A., Ziegler, L. E., Beresford, B., Mukherjee, S., Fraser, L., Fisher, V., O’Neill, M., Golder, S., Bedendo, A., & Taylor, J. (2023) · BMJ Supportive & Palliative Care · doi:10.1136/spcare-2022-004012 · Cited by 30
Stone, R., Walker, R., Marsh, N. M., & Ullman, A. J. (2023) · Australasian Emergency Care · doi:10.1016/j.auec.2023.06.001 · Cited by 38
“The quality assessment for each article, expressed as MMAT [23] is reported (see online Table 1).”
MethodGilroy, J., Veli‐Gold, S., Wright, W., Dew, A., Jensen, H., Bulkeley, K., & Lincoln, M. (2023) · Australian Journal of Rural Health · doi:10.1111/ajr.13020 · Cited by 23
Bingham, J., Kalu, F. A., & Healy, M. (2023) · Birth · doi:10.1111/birt.12759 · Cited by 36
Reasons that lead people to buy prescription medicines on the internet: a systematic review
Almomani, H., Raza, A., Patel, N., & Donyai, P. (2023) · Frontiers in Pharmacology · doi:10.3389/fphar.2023.1239507 · Cited by 22
Strandh, M. R., Hovén, E., Sörensdotter, R., Stålberg, K., Enebrink, P., Ljungman, L., & Wikman, A. (2023) · Critical Reviews in Oncology/Hematology · doi:10.1016/j.critrevonc.2023.104119 · Cited by 24
“To assess risk of bias in the included studies, the revised Mixed Methods Appraisal Tool (MMAT) was used (Hong et al., 2018, 2019).”
Uddin, T., Pitman, A., Benson, G., Kamal, Z., Hawton, K. E., & Rowe, S. L. (2023) · Psychological Medicine · doi:10.1017/s0033291723002805 · Cited by 31
“The MMAT has previously been validated for use in systematic reviews and was selected as it is designed to appraise a variety of study designs (Hong et al., 2018).”
MethodInfluentialBotschek, T., Hußlein, V., Peters, E. M. J., & Brosig, B. (2023) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2023.1227153 · Cited by 29
“To evaluate the risk of bias and the quality of the studies included, we used the Mixed methods appraisal tool [MMAT; (51)].”
MethodNewton, L., Keane, C. A., & Byrne, M. K. (2023) · Children and Youth Services Review · doi:10.1016/j.childyouth.2023.107368 · Cited by 31
Relationship between cyber and in-person dating abuse: A systematic review
Rodríguez‐deArriba, M., Santos, C., Cunha, O. S., Sánchez, V., & Caridade, S. (2024) · Aggression and Violent Behavior · doi:10.1016/j.avb.2024.101943 · Cited by 34
Teleoncology for children with cancer: A scoping review on applications and outcomes
Kermani, F., Orooji, A., & Sheikhtaheri, A. (2020) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2020.104118 · Cited by 24
Delemere, E., & Maguire, R. (2020) · Psycho-Oncology · doi:10.1002/pon.5542 · Cited by 26
“No overall scores are provided by the MMAT, rather interpretation took the following form, with 4–5 criteria met deemed high quality, 2–3 criteria met indicating moderate quality and 0–1 criteria deemed low quality, as per previous analysis using this tool.(37,38) No studies were excluded due to poor MMAT ratings.”
MethodCarandang, R. R., Sakamoto, J. L., Kunieda, M. K., Shibanuma, A., Львовна, Я. Е., Александровна, Б. М., & Jimba, M. (2021) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph18147463 · Cited by 38
“For non-RCTs, we used the following tools: ROBINS-I for quasi-experimental studies [21], NIH quality assessment tool for observational cohort and cross-sectional studies [22], critical appraisal skills program checklist for qualitative studies [23], and mixed methods appraisal tool for mixed-method studies [24].”
MethodShieu, B. M., Almusajin, J. A., Dictus, C., Beeber, A., & Anderson, R. A. (2021) · Journal of the American Medical Directors Association · doi:10.1016/j.jamda.2021.06.016 · Cited by 26
Engaging young people in climate change action: A scoping review of sustainability programs
Hohenhaus, M., Boddy, J., Rutherford, S., Roiko, A., & Hennessey, N. (2023) · Sustainability · doi:10.3390/su15054259 · Cited by 21
Moltrecht, B., Lange, A. M. C., Merrick, H., & Radley, J. (2024) · European Child & Adolescent Psychiatry · doi:10.1007/s00787-024-02380-3 · Cited by 31
“The Mixed Methods Appraisal Tool (MMAT) was used to assess the methodological quality of the included studies [11].”
MethodResultGross, J. L., & Koffman, J. (2024) · PLoS ONE · doi:10.1371/journal.pone.0299933 · Cited by 26
Help-seeking processes related to targeted school-based mental health services: systematic review
McPhail, L., Thornicroft, G., & Gronholm, P. C. (2024) · BMC Public Health · doi:10.1186/s12889-024-18714-4 · Cited by 49
“The Mixed Methods Appraisal Tool (MMAT) was employed to assess the methodological quality of included articles [37].”
BackgroundMethodInfluentialEustaquio, P. C., Smyth, J. M., & Salisi, J. A. (2024) · AIDS and Behavior · doi:10.1007/s10461-024-04393-0 · Cited by 26
“We performed the criti - cal appraisal of quality and risk of bias of included studies using the Mixed-Methods Appraisal Tool (MMAT) [30].”
Perona, M., Cooklin, A., Thorpe, C. M., O'Meara, P. F., & Rahman, M. A. (2024) · European Cardiology Review · doi:10.15420/ecr.2023.45 · Cited by 30
“Publication dates of articles included spanned 2000–23, with data collection periods from 1993 to 2019 (where reported).”
BackgroundCubelo, F., Jabri, F. A., Jokiniemi, K. S., & Turunen, H. E. (2024) · Scandinavian Journal of Caring Sciences · doi:10.1111/scs.13280 · Cited by 20
Hinata, S., Rohde, H., & Templeton, A. (2024) · International Journal of Disaster Risk Reduction · doi:10.1016/j.ijdrr.2024.104719 · Cited by 36
Support from Social Media during the COVID-19 Pandemic: A Systematic Review
Szeto, S., Au, A. K. Y., & Cheng, S. K. L. (2024) · Behavioral Sciences · doi:10.3390/bs14090759 · Cited by 36
“…Methods Assessment Tool (MMAT) is a quality assessment protocol designed specifically for systematic reviews of articles with qualitative, quantitative, and mixed method designs [58]. specifically for systematic reviews of articles with qualitative, quantitative, and mixed method designs [58].”
MethodHuff, N. R., Dunderdale, L., Kellogg, A. J., & Isbell, L. M. (2024) · Clinical Psychology Review · doi:10.1016/j.cpr.2024.102504 · Cited by 36
Hotte‐Meunier, A., Sarraf, L., Bougeard, A., Bernier, F., Voyer, C., Deng, J., Asmar, S. E., Σταμάτη, Α., Corbière, M., Villotti, P., et al. (2024) · Neurodiversity · doi:10.1177/27546330241287655 · Cited by 45
Redjem, I. D., Huaulmé, A., Jannin, P., & Michinov, E. (2025) · Nurse Education Today · doi:10.1016/j.nedt.2025.106583 · Cited by 37
Tan, N. Q. P., Nargund, R. S., Douglas, E. E., Lopez-Olivo, M. A., Resong, P. J., Ishizawa, S., Nofal, S., Krause, K. J., Volk, R. J., & Toumazis, I. (2025) · The Lancet Public Health · doi:10.1016/s2468-2667(24)00278-0 · Cited by 19
Shahnaz, I., Rose‐Clarke, K., Michelson, D., & Gronholm, P. C. (2025) · Health Expectations · doi:10.1111/hex.70204 · Cited by 23
Sunjaya, A. M., Schreiber, T., Kantilal, K., Davies, N., & Griffiths, S. (2025) · Age and Ageing · doi:10.1093/ageing/afaf120 · Cited by 34
“Mixed-method studies were assessed using the Mixed Methods Appraisal Tool (MMAT) [30].”
Use of augmented reality in human wayfinding: a systematic review
Qiu, Z., Mostafavi, A., & Kalantari, S. (2025) · Virtual Reality · doi:10.1007/s10055-025-01226-w · Cited by 37
“To evaluate the quality of the included articles, we used the Mixed Methods Appraisal Tool (MMAT) within the Covidence platform (Hong et al. 2018).”
Dodd, M., Ivers, R., Zwi, A. B., Rahman, A., & Jagnoor, J. (2019) · Health Policy and Planning · doi:10.1093/heapol/czz044 · Cited by 26
“Therefore, the MMAT was a useful tool to identify the study design and to critically appraise the coherence between data sources, collection, synthesis and interpretation (Hong et al., 2018).”
MethodMwendwa, P., LAWLOR, B. A., Kroll, T., & Brún, A. D. (2022) · BMC Geriatrics · doi:10.1186/s12877-022-03668-2 · Cited by 30
“Abbreviations AD: Alzheimer’s disease; HIC: High-income countries; DATAD: Database of African Theses and Dissertations; LMIC: Low- and middle-income countries; MMAT: Mixed Methods Appraisal Tool; PLWD: People living with dementia; PRISMA: Preferred Reporting Items for Systematic Reviews and Meta-Analysis; SSA: Sub-Sah…”
MethodInfluentialLara-Bocanegra, A., Gómez-Millán, M. R. B., & Fernández, J. G. (2021) · International Journal of Sports Marketing and Sponsorship · doi:10.1108/ijsms-12-2020-0212 · Cited by 29
Paramasivan, S., Davies, P. A., Richards, A., Wade, J., Rooshenas, L., Mills, N., Realpe, A. X., Raj, J. P., Subramani, S., Ives, J., et al. (2021) · BMJ Global Health · doi:10.1136/bmjgh-2020-004729 · Cited by 20
Laing, B. B., & Crowley, J. (2021) · Nurse Education in Practice · doi:10.1016/j.nepr.2021.103137 · Cited by 23
Grünwald, L. M., Duddy, C., Byng, R., Crellin, N. E., & Moncrieff, J. (2021) · BMC Psychiatry · doi:10.1186/s12888-021-03355-3 · Cited by 22
Guggisberg, M., Bottino, S., & Doran, C. M. (2021) · Frontiers in Psychology · doi:10.3389/fpsyg.2021.699288 · Cited by 39
“Before articles were included in the final review, an independent quality appraisal by the first and second author was conducted utilizing the standardized quality assessment Mixed Methods Appraisal Tool (MMAT) for systematic reviews (Hong et al., 2018), which has showed content validity and reliability (Hong et al.”
MethodFetta, J., Starkweather, A. R., Huggins, R., Hoof, T. V., Casa, D. J., & Gill, J. (2021) · The Journal of School Nursing · doi:10.1177/10598405211056646 · Cited by 20
“An a priori quality appraisal, a score on the MMAT of 75% or higher was used to select publications for the final analysis, meaning that there could be one noted area of weakness in the design or methods of the study that could potentially bias the results.”
MethodInfluentialFernandes, D. V., Martins, A. R., Canavarro, M. C., & Moreira, H. (2021) · Journal of Child and Family Studies · doi:10.1007/s10826-021-02175-z · Cited by 28
“To assess the risk of bias of empirical studies, we used the Mixed-Methods Appraisal Tool (MMAT) (Hong et al., 2018), which is a critical appraisal tool designed for the appraisal stage of reviews that include qualitative, quantitative and mixed-methods studies.”
BackgroundMethodLin, T. K., Werner, K., Witter, S., et al. (2022) · Health Policy · doi:10.1016/j.healthpol.2022.03.016 · Cited by 20
Hanley, E., Martin, A., Dalton, C., & Lehane, E. A. (2022) · Journal of Intellectual Disabilities · doi:10.1177/17446295221115914 · Cited by 31
“The scoring system was removed from the 2018 version of the MMAT as it is advised to provide a more detailed presentation of the ratings of each criterion (Hong et al., 2018).”
MethodVu, M. N. L., Do, A. L., Boyer, L., Tran, Q. C., Kohler, S., Ahmed, S. I., Molnar, A., Vu, T. S., Vo, N. T. H., Nguyen, L. M. V., et al. (2022) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph191811612 · Cited by 27
“The methodological quality of the studies was assessed by using the Mixed Methods Appraisal Tool, which examines the methodological quality of qualitative research, randomized controlled trials, nonrandomized studies, quantitative descriptive studies, and mixed-methods studies [31].”
MethodJaiswal, A., Gupta, S., Paramasivam, A., Santhakumaran, S., Holzhey, P., Dupont, P., & Wittich, W. (2022) · Innovation in Aging · doi:10.1093/geroni/igac076 · Cited by 23
“…(for all types of studies) followed by five criteria questions for methodological quality; a total of 25 criteria on five categories of studies—qualitative research, randomized controlled trials, nonrandomized studies, quantitative descriptive studies, and mixed methods studies (Hong et al., 2018).”
BackgroundMethodInfluentialDunbar, P., Keyes, L. M., & Browne, J. P. (2023) · PLoS ONE · doi:10.1371/journal.pone.0278007 · Cited by 57
“Quality appraisal of quantitative and qualitative studies used the Joanna Briggs Institute tools [28]; for mixed-methods studies we used the Mixed-Methods Appraisal Tool (MMAT) [29].”
MethodPromoting sleep in hospitals: An integrative review of nurses' attitudes, knowledge and practices
Mohedat, H., & Somayaji, D. (2023) · Journal of Advanced Nursing · doi:10.1111/jan.15694 · Cited by 23
Soon, L., McDowall, A., & Teoh, K. (2023) · Psychiatry Psychology and Law · doi:10.1080/13218719.2023.2206879 · Cited by 40
Healthcare resource use and costs of varicella and its complications: A systematic literature review
Williame, I., George, M., Shah, H. A., et al. (2023) · Human Vaccines & Immunotherapeutics · doi:10.1080/21645515.2023.2266225 · Cited by 20
“The quality of studies was assessed using the mixed meth-ods appraisal tool (MMAT) 13 for quantitative and qualitative studies and the Consolidated Health Economic Evaluation Reporting Standards (CHEERS) checklist in case of cost-effectiveness publications.”
MethodRahman, S. A. F. S. A., Maulud, K. N. A., Ujang, U., Jaafar, W. S. W. M., Shaharuddin, S., & Rahman, A. A. A. (2024) · SAGE Open · doi:10.1177/21582440231220768 · Cited by 35
“The Syed Abdul Rahman et al. Mixed-Method Appraisal Tool (MMAT), recommended by Hong et al. (2018), was employed for this purpose.”
A systematic review of Earthquake Early Warning (EEW) systems based on Artificial Intelligence
Kolivand, P., Saberian, P., Tanhapour, M., et al. (2024) · Earth Science Informatics · doi:10.1007/s12145-024-01253-2 · Cited by 20
“In order to evaluate the methodological quality of the studies included in this review, the Mixed Methods Assessment Tool (MMAT) version 2018 (Hong et al. 2018) was employed.”
Barriers and facilitators to patient education provided by nurses: A mixed‐method systematic review
Wang, S., Liu, K., Tang, S., Wang, G., Qi, Y., & Chen, Q. (2024) · Journal of Clinical Nursing · doi:10.1111/jocn.17111 · Cited by 26
A systematic review of online team based learning approaches in health professional education
Burton, R., Kellett, U., Mansah, M., & Sriram, D. (2024) · Nurse Education Today · doi:10.1016/j.nedt.2024.106156 · Cited by 21
Factors influencing thirst in ICU patients: A mixed methods systematic review
Xie, X.-M., Huang, D., Chun, S., Bai, D.-X., Lu, X., Li, Y., Hou, C., Ji, W., & Gao, J. (2024) · Intensive and Critical Care Nursing · doi:10.1016/j.iccn.2024.103811 · Cited by 22
Impacts of dance on agitation and anxiety among persons living with dementia: An integrative review
Bennett, C. G., Fox, H., McLain, M., & Medina-Pacheco, C. (2020) · Geriatric Nursing · doi:10.1016/j.gerinurse.2020.07.016 · Cited by 29
A Mixed-Studies Review of the School-to-Prison Pipeline and a Call to Action for School Nurses
Aronowitz, S. V., Kim, B., & Aronowitz, T. B. (2020) · The Journal of School Nursing · doi:10.1177/1059840520972003 · Cited by 25
“The present version of the tool was developed on the basis of findings from a literature review of critical appraisal tools, interviews with MMAT users, and an e-Delphi study with international experts (Hong et al., 2018; Hong, Gonzalez-Reyes, & Pluye, 2018).”
MethodCatoms: Moving robots without moving parts
Kirby, B., Campbell, J. D., Aksak, B., Pillai, P. S., Hoburg, J. F., Mowry, T. C., & Goldstein, S. C. (1999) · Cited by 34
Fisette-Paulhus, I., Gagnon, C., Girard‐Côté, L., & Morin, M. (2022) · Neuromuscular Disorders · doi:10.1016/j.nmd.2022.01.008 · Cited by 21
Mental health consumers' perspectives of physical health interventions: An integrative review
Tabvuma, T., Stanton, R., Browne, G., & Happell, B. (2022) · International Journal of Mental Health Nursing · doi:10.1111/inm.13000 · Cited by 29
Camouflaging in autism: a systematic reviewPreprint
Cook, J. M., Hull, L., Crane, L., & Mandy, W. (2021) · OSF Preprints (OSF Preprints) · doi:10.31219/osf.io/u5b9e · Cited by 29
The Psychosocial Impact of COVID-19 on Older Adults with Cancer: A Rapid Review
Verma, R., Kilgour, H. M., & Haase, K. R. (2022) · Current Oncology · doi:10.3390/curroncol29020053 · Cited by 19
“Older adults demonstrated resilience by maintaining social connectedness with family and friends either virtually [17] or through physically distant interactions [19].”
BackgroundMethodVolpato, E., Cavalera, C., Castelnuovo, G., Molinari, E., & Pagnini, F. (2022) · BMC Psychiatry · doi:10.1186/s12888-022-03902-6 · Cited by 33
“The methodological quality of each study included in the systematic review was assessed using a standard form adapted from the Mixed Methods Appraisal Tool (MMAT) –Version 2018, a critical appraisal tool that is designed for the appraisal stage of systematic mixed studies reviews [44].”
MethodA systematic review of climate emotions and mental health in adults
Pitt, C., Norris, K., & Pecl, G. (2023) · Global Environmental Psychology · doi:10.5964/gep.11405 · Cited by 20
“Conversely, studies of lower quality (60% MMAT score or below) were more likely to suggest no relationship between climate emotions and negative mental health.”
MethodInfluentialGrief and bereavement support for parents in low- or middle-income countries: a systematic review
Pedraza, E. C., Vokinger, A. K., Cleves, D., Michel, G., Wrigley, J., Baker, J. N., Garcia-Quintero, X., & McNeil, M. J. (2024) · Journal of Pain and Symptom Management · doi:10.1016/j.jpainsymman.2024.01.023 · Cited by 19
Röscher, P., Sathiram, R., Milios, J. E., et al. (2022) · Systematic Reviews · doi:10.1186/s13643-021-01865-5 · Cited by 19
“The MMAT tool version 2011 was used to assess the methodological quality of the included studies.”
BackgroundMethodInfluentialPatra, E., Κοκκινοπούλου, Ά., & Pagkalos, I. G. (2023) · Nutrients · doi:10.3390/nu15112460 · Cited by 19
“The latest versio of the Mixed Methods Appraisal Tool (MMAT) was used for quality assessment of all included studies [52].”
MethodInterprofessional supervision among allied health professionals: a systematic scoping review
McGuinness, S., & Guérin, S. (2024) · Journal of Interprofessional Care · doi:10.1080/13561820.2024.2343837 · Cited by 19
“In line with instructions for utilizing the MMAT for quality appraisal (Hong et al., 2018), the authors rated a number of criteria for each study in the areas of study design, data collection, analysis and reporting using ratings of “yes,” “no,” or “can’t tell.””
MethodInfluential
Metadata
- Authors
- Hong, Q. N., Fàbregues, S., Bartlett, G., Boardman, F., Cargo, M., Dagenais, P., Gagnon, M.-P., Griffiths, F., Nicolau, B., O’Cathain, A., Rousseau, M.-C., Vedel, I., & Pluye, P.
- Affiliations
- McGill University (CA)
- Universitat Oberta de Catalunya (ES)
- University of Warwick (GB)
- University of Canberra (AU)
- Université de Sherbrooke (CA)
- Université Laval (CA)
- University of Sheffield (GB)
- Institut National de la Recherche Scientifique (CA)
- Corresponding author
- Pluye, Pierre
- ORCID
- Venue
- Education for Information
- Publisher
- SAGE Publications
- Published
- 2018-12-18
- Volume
- 34
- Issue
- 4
- Pages
- 285-291
- ISSN
- 0167-8329 (ISSN-L)
- 1875-8649
- OpenAlex
- W2901724277
- Cited by
- 4,737
- References
- 21
- Citations by year
- 2026: 1312 · 2025: 974 · 2024: 800 · 2023: 610 · 2022: 501 · 2021: 370 · 2020: 143 · 2019: 22 · 2018: 3
- FWCI
- 102.21
- Type
- journal article
- Language
- en
- Open access
- green
- hdl.handle.net
- Licence
- cc-by-nc-nd
- Locations
- Education for Information (publishedVersion)
- O2 - Repositori Institucional (Universitat Oberta de Catalunya) (publishedVersion, cc-by-nc-nd)
- RECERCAT (Consorci de Serveis Universitaris de Catalunya) (publishedVersion, cc-by-nc-nd)
- White Rose Research Online (University of Leeds, The University of Sheffield, University of York) (acceptedVersion)
- Warwick Research Archive Portal (University of Warwick) (acceptedVersion)
- HAL (submittedVersion)
- eScholarship@McGill (McGill) (submittedVersion)
- LA Referencia (Red Federada de Repositorios Institucionales de Publicaciones Científicas) (submittedVersion, cc-by-nc-nd)
- eScholarship@McGill (McGill) (submittedVersion, cc-by-nc-nd)
- EspaceINRS (National Institute for Scientific Research (Canada)) (submittedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Sociology and Political Science › Delphi Technique in Research
- Social Sciences › Education › Qualitative Research Methods and Applications
- Keywords
- Mixed Methods Appraisal Tool, critical appraisal, study quality assessment, mixed methods, systematic reviews, qualitative research, quantitative research, randomized controlled trials, non-randomized studies, e-Delphi, inter-rater reliability
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.