The Mixed Methods Appraisal Tool (MMAT) version 2018 for information professionals and researchers
Open accessThe DOI matches its Crossref record; 21 of the 21 references it cites were found in a registry or an index; 2 of the 26 sources its claims cite were checked for support.
Abstract
INTRODUCTION: Appraising the quality of studies included in systematic reviews combining qualitative and quantitative evidence is challenging. To address this challenge, a critical appraisal tool was developed: the Mixed Methods Appraisal Tool (MMAT). The aim of this paper is to present the enhance ments made to the MMAT. DEVELOPMENT: The MMAT was initially developed in 2006 based on a literature review on systematic reviews combining qualitative and quantitative evidence. It was subject to pilot and interrater reliability testing. A revised version of the MMAT was developed in 2018 based on the results from usefulness testing, a literature review on critical appraisal tools and a modified e-Delphi study with methodological experts to identify core criteria. TOOL DESCRIPTION: The MMAT assesses the quality of qualitative, quantitative, and mixed methods studies. It focuses on methodological criteria and includes five core quality criteria for each of the following five categories of study designs: (a) qualitative, (b) randomized controlled, (c) nonrandomized, (d) quantitative descriptive, and (e) mixed methods. CONCLUSION: The MMAT is a unique tool that can be used to appraise the quality of different study designs. Also, by limiting to core criteria, the MMAT can provide a more efficient appraisal.
Overview
CoChat Summary
TL;DR
This paper presents version 2018 of the Mixed Methods Appraisal Tool (MMAT), a single critical appraisal tool for assessing the methodological quality of qualitative, quantitative, and mixed methods studies in mixed studies reviews. It describes the tool's 25 criteria across five study design categories, its development history, and guidance for use.
Key findings
The MMAT version 2018 includes a total of 25 criteria and 2 screening questions, covering five categories of study designs: (a) qualitative, (b) randomized controlled trial, (c) nonrandomized, (d) quantitative descriptive, and (e) mixed methods, with five core criteria per category rated as yes, no, or can't tell. The tool was developed from a 2006 first version (15 criteria across four categories), revised in 2011, and further revised into the 2018 version based on usefulness testing with 20 researchers, a literature review on critical appraisal tools, and a modified e-Delphi study with 73 methodological experts. Two interrater reliability studies used 32 and 261 papers respectively and showed the need to clarify criteria, particularly for nonrandomized and qualitative studies. The paper argues the MMAT offers a time-efficient alternative by limiting to core criteria and uniquely includes specific criteria for mixed methods studies, though its criteria are harder to judge because they target methodological rather than reporting quality.
Methods
This is a tool description paper (third in a three-part series on systematic reviews for information professionals) rather than an empirical study; it narrates the development of the MMAT and explains how to apply it. Development drew on a literature review on mixed studies reviews, pilot testing in workshops with four reviewers appraising six studies, two interrater reliability studies (32 and 261 papers), usefulness testing via interviews with 20 researchers, a literature review on critical appraisal tools, and a modified e-Delphi study with 73 methodological experts. Use involves three steps: two optional screening questions, selection of the appropriate design category (or categories), and rating each criterion as yes, no, or can't tell, with detailed presentation of ratings advised over an overall numerical score.
Limitations
The paper states that revision of the MMAT has so far focused on content validity and usefulness, and that further testing on its validity and reliability will be needed; it also notes that modifications may be necessary as evidence develops. It states that MMAT criteria are more difficult to judge than other appraisal tools because they focus on methodological quality rather than reporting quality. Evident from the text: the tool's development evidence base is described only in summary form, with reliability and validity results reported by reference to prior studies rather than presented here.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
2 Valid · 0 Not valid · 18 Not checked · 11 Own results and interpretations
Every source the paper cites for this claim was checked and supports it.
In the paper
“To further its development, usefulness testing was performed by interviewing 20 researchers who have used the MMAT in a systematic review or have contacted the developer of the tool for questions or permission to use the tool (18).”
What the sources say
- Supportedcites [18] 18. Hong QN, Gonzalez-Reyes A, Pluye P. Improving the usefulness of a tool for appraising the quality of qualitative, quantitative and mixed methods studies, the Mixed Methods Appraisal Tool (MMAT). J Eval Clin Pract. 2018;24(3):459-67. source
The Abstract reports semi-structured interviews with 20 participants to explore the MMAT’s usefulness. The Study Participants section says researchers were recruited through systematic reviews that used the MMAT and a list of people who had contacted its developer for permission or guidance. (LLM, conf=0.94)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“The results of this study were helpful to identify changes to be made in the MMAT.”
What the sources say
- Supportedcites [18] 18. Hong QN, Gonzalez-Reyes A, Pluye P. Improving the usefulness of a tool for appraising the quality of qualitative, quantitative and mixed methods studies, the Mixed Methods Appraisal Tool (MMAT). J Eval Clin Pract. 2018;24(3):459-67. source
The Discussion says the study explored MMAT users’ experiences to help revise the tool and identified themes suggesting areas for improvement. The Recommendations for Improving the MMAT then proposes specific changes on the basis of those results. (LLM, conf=0.94)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
In the paper
“Moreover, it includes specific criteria for mixed methods studies, which is not often found in other tools (21).”
What the sources say
- Not matchedcites [21] 21. Pluye P. Critical appraisal tools for assessing the methodological quality of qualitative, quantitative and mixed methods studies included in systematic mixed studies reviews. J Eval Clin Pract. 2013;19(4):722.
We searched by its title and found no matching paper.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Not matched
In the paper
“To address this challenge, a critical appraisal tool for assessing the quality of quantitative, qualitative and mixed methods studies was developed: the Mixed Methods Appraisal Tool (MMAT) (7).”
What the sources say
- Not checked — no full text of the sourcecites [7] 7. Pluye P, Gagnon MP, Griffiths F, Johnson-Lafleur J. A scoring system for appraising mixed methods research, and concomitantly appraising qualitative, quantitative and mixed methods primary studies in mixed studies reviews. Int J Nurs Stud. 2009;46(4):529-46. source
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“Moreover, it includes specific criteria for mixed methods studies, which is not often found in other tools (21).”
What the sources say
- Not matchedcites [21] 21. Pluye P. Critical appraisal tools for assessing the methodological quality of qualitative, quantitative and mixed methods studies included in systematic mixed studies reviews. J Eval Clin Pract. 2013;19(4):722.
We searched by its title and found no matching paper.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Not matched
5 more claims
Create a free account to see all 31 claims and to verify the 1 reference still open.
- It may state two separate claims as one. 7
- Probably not a substantial claim. 16
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Quality was assessed using two validated tools by study design. Quantitative studies were appraised using the Mixed Methods Appraisal Tool Version 2018, which evaluates five methodological criteria on a binary scale (Hong et al., 2018). Qualitative studies were additionally appraised using the Critical Appraisal Skills Programme Qualitative Checklist, which evaluates 10 criteria (CASP UK, 2018).”
From Cultural stigma and caregiver burden among family caregivers of people with nervous system disorders in the Middle East and North Africa: A mixed-methods systematic review (2026) · cites this paper as Hong et al., 2018
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“The quality of the included studies was assessed using the 2018 version of the Mixed Methods Appraisal Tool (MMAT).15”
From Psychosocial interventions for siblings of children with cancer: A mixed methods systematic review (2021) · cites this paper as [15]
The cited source says what the claim attributes to it.
In the citing paper
“To further limit risks of bias, a formal risk of bias assessment tool (ROB-tool), the Mixed Methods Appraisal Tool [52], was used to evaluate the overall study quality.”
From Mediators of outcome in adolescent psychotherapy and their implications for theories and mechanisms of change: a systematic review (2024) · cites this paper as [52]
The cited source says what the claim attributes to it.
In the citing paper
“LM, with the support of a research assistant, assessed the methodological quality of empirical studies (qualitative, quantitative and mixed methods) using the Mixed Methods Appraisal Tool (MMAT) [45].”
From Transnationalism and care of migrant families during pregnancy, postpartum and early-childhood: an integrative review (2020) · cites this paper as [45]
The cited source says what the claim attributes to it.
In the citing paper
“The quality of the included studies was independently assessed by two authors using the Mixed Methods Appraisal Tool (MMAT) (Hong et al., 2018).Disagreements were resolved through discussion.”
From Gamification on Mathematics Engagement and Motivation in Secondary School and Higher Education: A Systematic Review and Meta-Analysis (2026) · cites this paper as Hong et al., 2018
29 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ 4. Conclusion · p. 5
“## 4. Conclusion”
§ 4. How to use the MMAT? · p. 4
“## 4. How to use the MMAT?”
Citations
Cites
Pace, R., Pluye, P., Bartlett, G., Macaulay, A. C., Salsberg, J., Jagosh, J., & Seller, R. (2012) · International Journal of Nursing Studies · doi:10.1016/j.ijnurstu.2011.07.002 · Cited by 1,331 · cited 2×
“Then, two interrater reliability studies on the MMAT were conducted using respectively 32 and 261 papers (Pace et al., 2012; Souto et al., 2015).”
Self-citationPluye, P., Gagnon, M.-P., Griffiths, F., & Johnson-Lafleur, J. (2009) · International Journal of Nursing Studies · doi:10.1016/j.ijnurstu.2009.01.009 · Cited by 1,119 · cited 3×
“This led to develop a first version of the MMAT that is in line with a social constructionist worldview (Pluye et al., 2009).”
InfluentialSelf-citationHong, Q. N., Gonzalez‐Reyes, A., & Pluye, P. (2018) · Journal of Evaluation in Clinical Practice · doi:10.1111/jep.12884 · Cited by 1,107 · cited 3×
“To further its development, usefulness testing was performed by interviewing 20 researchers who have used the MMAT in a systematic review or had contacted the developer of the tool for questions or permission to use the tool (Hong et al., 2018a).”
Self-citationSystems to rate the strength of scientific evidence.
West, S. L., King, V., Carey, T. S., Lohr, K. N., Mckoy, N. A., Sutton, S. F., & Lux, L. J. (2002) · PubMed Central · Cited by 923
Zaza, S., Wright-De Agüero, L. K., Briss, P. A., Truman, B. I., Hopkins, D. P., Hennessy, M. H., Sosin, D. M., Anderson, L., Carande-Kulis, V. G., Teutsch, S. M., et al. (2000) · American Journal of Preventive Medicine · doi:10.1016/s0749-3797(99)00122-1 · Cited by 464 · cited 1×
“The algorithm was developed based on several existing algorithms of quantitative study designs (Hartling et al., 2010; Hartling et al., 2011; National Institute for Health Care Excellence, 2012; Scottish Intercollegiate Guidelines Network; Seo et al., 2016; West et al., 2002; Zaza et al., 2000).”
Souto, R. Q., Khanassov, V., Hong, Q. N., Bush, P. L., Vedel, I., & Pluye, P. (2015) · International Journal of Nursing Studies · doi:10.1016/j.ijnurstu.2014.08.010 · Cited by 375 · cited 1×
“Then, two interrater reliability studies on the MMAT were conducted using respectively 32 and 261 papers (Pace et al., 2012; Souto et al., 2015).”
Self-citationA review of critical appraisal tools show they lack rigor: Alternative tool structure is proposed
Crowe, M., & Sheppard, L. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2010.02.008 · Cited by 261 · cited 2×
“Currently, there exist more than 500 critical appraisal tools (i.e., checklists including a list of criteria to judge the quality of a study) (Bai et al., 2012; Crowe & Sheppard, 2011; Heyvaert et al., 2013; Santiago-Delefosse et al., 2016).”
Carroll, C., Booth, A., & Lloyd-Jones, M. (2012) · Qualitative Health Research · doi:10.1177/1049732312452937 · Cited by 248 · cited 1×
“Methodological quality criteria are more difficult to interpret because the reviewers need to judge whether the reported results of a study are trustworthy (Carroll et al., 2012; Hong & Pluye, 2018a).”
Adjustment of meta-analyses on the basis of quality scores should be abandoned
Herbison, P., Hay-Smith, J., & Gillespie, W. J. (2006) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2006.03.008 · Cited by 207 · cited 1×
“Currently in the literature, it is discouraged to calculate an overall score from the ratings of each criterion (Herbison et al., 2006; Higgins & Green, 2008; Viswanathan et al., 2012).”
Critical Appraisal of Mixed Methods Studies
Heyvaert, M., Hannes, K., Maes, B., & Onghena, P. (2013) · Journal of Mixed Methods Research · doi:10.1177/1558689813479449 · Cited by 168 · cited 1×
“Currently, there exist more than 500 critical appraisal tools (i.e., checklists including a list of criteria to judge the quality of a study) (Bai et al., 2012; Crowe & Sheppard, 2011; Heyvaert et al., 2013; Santiago-Delefosse et al., 2016).”
Cited by
Items 401–500 of 4,578
do Amaral, J. M. X., Kimergård, A., & Deluca, P. (2022) · BMJ Open · doi:10.1136/bmjopen-2021-056445 · Cited by 50
Hall, C. E., Davidson, L., Brooks, S. K., Greenberg, N., & Weston, D. (2023) · BMC Psychology · doi:10.1186/s40359-023-01221-3 · Cited by 39
Hada, A., & Coyer, F. M. (2021) · Nursing and Health Sciences · doi:10.1111/nhs.12825 · Cited by 64
Huang, A., Hong, W., Zhao, B., Lin, J., Rui, X., & Wang, Y. (2022) · Nursing Open · doi:10.1002/nop2.1384 · Cited by 39
“The Mixed Methods Appraisal Tool (MMAT) was applied by two authors to independently assess the methodological quality of included studies (Table 2) (Hong et al., 2018).”
BackgroundMethodInfluentialFactors related to readiness for practice among undergraduate nursing students: A systematic review
Lee, T. W., Damiran, D., Konlan, K. D., Ji, Y., Yoon, Y. S., & Ji, H. (2023) · Nurse Education in Practice · doi:10.1016/j.nepr.2023.103614 · Cited by 49
The use of heart rate variability in esports: A systematic review
Welsh, M. R., Mosley, E., Laborde, S., Day, M. C., Sharpe, B. T., Burkill, R. A., & Birch, P. D. J. (2023) · Psychology of sport and exercise · doi:10.1016/j.psychsport.2023.102495 · Cited by 58
“Both S. J. Hong et al. (2018) and D. Lee et al. (2021) suggest the severity of IGD predicted a reduction in LnHF (uns).”
InfluentialPsychological Factors Related to Treatment Outcomes in Head and Neck Cancer
Mäkitie, A. A., Alabi, R. O., Pulkki-Råbäck, L., Almangush, A., Beitler, J. J., Saba, N. F., Strojan, P., Takes, R. P., Guntinas‐Lichius, O., & Ferlito, A. (2024) · Advances in Therapy · doi:10.1007/s12325-024-02945-3 · Cited by 52
“(Table 2) [12, 13].”
Effectiveness of 30 km/h speed limit – A literature review
Yannis, G., & Michelaraki, E. (2024) · Journal of Safety Research · doi:10.1016/j.jsr.2024.11.003 · Cited by 43
Plessas, A., Paisi, M., Ahmed, N., Brookes, Z. L. S., Burns, L., & Witton, R. (2024) · BMC Medical Education · doi:10.1186/s12909-024-06367-1 · Cited by 75
“All studies regardless of methodological quality were included in the review [26].”
Gautam, S., Neville, S. J., & Montayre, J. (2019) · International Journal of Older People Nursing · doi:10.1111/opn.12228 · Cited by 45
Mayland, C. R., Powell, R. A., Clarke, G., Ebenso, B., & Allsop, M. J. (2021) · PLoS ONE · doi:10.1371/journal.pone.0252188 · Cited by 39
“Due to the heterogeneity of study types, the Mixed Method Appraisal Tool (MMAT) was used for assessing the quality of both quantitative and qualitative studies [17].”
MethodTaheri, S., Sichani, M. G., & Shabani, A. (2021) · Social Science & Medicine · doi:10.1016/j.socscimed.2021.113820 · Cited by 46
Franc, J. M., Kirkland, S. W., Wisnesky, U. D., Campbell, S., & Rowe, B. H. (2021) · Prehospital and Disaster Medicine · doi:10.1017/s1049023x2100131x · Cited by 44
Bogren, M. U., Alesö, A., Teklemariam, M., Sjöblom, H., Hammarback, L., & Erlandsson, K. (2021) · Women and Birth · doi:10.1016/j.wombi.2021.06.006 · Cited by 53
“The MMAT is a critical appraisal tool that includes 25 criteria in fi ve study categories that include qualitative, quantitative, and mixed-methods studies [13].”
MethodAdult Inpatients’ Perceptions of Their Fall Risk: A Scoping Review
Dabkowski, E., Cooper, S. J., Duncan, J. R., & Missen, K. (2022) · Healthcare · doi:10.3390/healthcare10060995 · Cited by 40
“evaluated using the Mixed Methods Appraisal Tool (MMAT) [20].”
MethodInfluentialSchmidt, F., Varese, F., Larkin, A., & Bucci, S. (2024) · Trauma, Violence, & Abuse · doi:10.1177/15248380231207896 · Cited by 39
“However, due to the variety of study types and the different instruments to both measure the experiences of NCSSI and the mental health impact, direct comparison of the quality is difficult (Hong et al., 2018).”
BackgroundSchofield, R., Dibb, B., Coles-Gale, R., & Jones, C. J. (2021) · International Journal of Nursing Studies · doi:10.1016/j.ijnurstu.2021.103927 · Cited by 37
Koren, Y., Leveille, S. G., & You, T. (2021) · Research in Gerontological Nursing · doi:10.3928/19404921-20210325-02 · Cited by 45
Zhao, I. Y., Ma, Y. X., Yu, M. W. C., Liu, J., Dong, W., Pang, Q., Lu, X. Q., Molassiotis, A., Holroyd, E. A., & Wong, W. C. W. (2021) · Journal of Medical Internet Research · doi:10.2196/32328 · Cited by 49
“We evaluated study quality and methodological rigor for 5 empirical studies by using the modified mixed-methods appraisal tool (MMAT) [14], which was not applicable for the remaining discussion papers.”
Health system governance in settings with conflict-affected populations: a systematic review
Lokot, M., Bou-Orm, I. R., Zreik, T., Kik, N., Fuhr, D. C., Masri, R. E., Meagher, K., Smith, J., Asmar, M. K., McKee, M., et al. (2022) · Health Policy and Planning · doi:10.1093/heapol/czac027 · Cited by 39
Foroughi, Z., Ebrahimi, P., Aryankhesal, A., Maleki, M., & Yazdani, S. (2022) · BMC Health Services Research · doi:10.1186/s12913-022-08316-4 · Cited by 51
“, studies with qualitative, quantitative, and mixedmethod designs) [19, 20].”
BackgroundAn integrative review of supportive relationships between child‐bearing women and midwives
Morbaty, H. Y. A., Ebert, L. M., Dowse, E., & Chan, S. W. (2022) · Nursing Open · doi:10.1002/nop2.1447 · Cited by 52
Galvin, E., Desselle, S. P., Gavin, B. E., Quigley, E., Flear, M. L., Kilbride, K., McNicholas, F., Cullinan, S., & Hayden, J. C. (2022) · Journal of Psychiatric Research · doi:10.1016/j.jpsychires.2022.10.051 · Cited by 50
Gonsalves, P. P., Nair, R., Roy, M., Pal, S., & Michelson, D. (2023) · Administration and Policy in Mental Health and Mental Health Services Research · doi:10.1007/s10488-023-01253-2 · Cited by 58
Ezaydi, N., Sheldon, E., Kenny, A., Buck, E. T., & Weich, S. (2023) · Health Expectations · doi:10.1111/hex.13788 · Cited by 50
“3 | Quality assessment Quality assessment was carried out using the Mixed Methods Appraisal Tool version 2018, to assess the methodological quality of eligible studies (Supporting Information: FS4).(14) This tool is designed for systematic reviews which incorporate quantitative, qualitative and mixed methods studies a…”
MethodMini review: Challenges in EEG emotion recognition
Zhang, Z., Mir, J. M. F., & Mateu, L. G. (2024) · Frontiers in Psychology · doi:10.3389/fpsyg.2023.1289816 · Cited by 51
Tele-mentoring using augmented reality technology in healthcare: A systematic review
Bui, D. T., Barnett, T., Hoang, H., & Chinthammit, W. (2021) · Australasian Journal of Educational Technology · doi:10.14742/ajet.6243 · Cited by 48
“The mixed method appraisal tool (MMAT; Hong et al., 2018) is a critical appraisal tool designed for the appraisal stage of systematic reviews, for example, reviews include qualitative, quantitative and mixed methods studies.”
MethodAccuracy of Wearable Transdermal Alcohol Sensors: Systematic Review
Brobbin, E., Deluca, P., Hemrage, S., & Drummond, C. (2022) · Journal of Medical Internet Research · doi:10.2196/35178 · Cited by 46
“With the MMAT, exclusion of low–methodological-quality studies is discouraged [53].”
BackgroundMethodBias Mitigation in Primary Health Care Artificial Intelligence Models: Scoping Review
Sasseville, M., Ouellet, S., Rhéaume, C., Sahlia, M., Couture, V., Després, P., Paquette, J.-S., Darmon, D., Bergeron, F., & Gagnon, M.-P. (2025) · Journal of Medical Internet Research · doi:10.2196/60269 · Cited by 38
“One senior reviewer appraised the quality of the included studies by applying the Mixed-Methods Appraisal Tool (MMAT) [22,23] and at least one senior researcher validated each of them.”
Self-citationAlkhaldi, O., McMillan, B., Maddah, N., & Ainsworth, J. (2023) · JMIR mHealth and uHealth · doi:10.2196/43561 · Cited by 37
“The MMAT is a comprehensive tool for assessing the quality of quantitative, qualitative, and mixed methods study designs [21].”
MethodOkeke, E. S., Ezeorba, T. P. C., Chen, Y., Mao, G., Feng, W., & Wu, X. (2022) · Environmental Science and Pollution Research · doi:10.1007/s11356-022-22612-w · Cited by 37
Socio-economic drivers of drug-resistant tuberculosis in Africa: a scoping review
Cannon, L.-A. L., Oladimeji, K. E., & Goon, D. T. (2021) · BMC Public Health · doi:10.1186/s12889-021-10267-0 · Cited by 38
I think I'm sleepy, therefore I am – Awareness of sleepiness while driving: A systematic review
Cai, A. W. T., Manousakis, J. E., Lo, T., Horne, J. A., Howard, M. E., & Anderson, C. (2021) · Sleep Medicine Reviews · doi:10.1016/j.smrv.2021.101533 · Cited by 55
Patients' Health & Well-Being in Inpatient Mental Health-Care Facilities: A Systematic Review
Weber, C., Flores, V. M., Wheele, T. P., Miedema, E., & White, E. (2022) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2021.758039 · Cited by 49
“Study type Target population Participants & sample size Methods/Measure Qual Building/design feature Domain Outcome Smith and Jones, 2014, UK (88) Mixed-methods, sequential explanatory design (quantitative non-randomized / descriptive analysis; qualitative phenomenological) Patients in PICU (acute disturbed phase, hig…”
BackgroundMethodInfluentialSelf-management interventions to improve mobility after stroke: an integrative review
Sahely, A., Giles, D., Sintler, C., Soundy, A., & Rosewilliam, S. (2022) · Disability and Rehabilitation · doi:10.1080/09638288.2022.2028019 · Cited by 42
Factors preceding occupational distress in emergency nurses: An integrative review
Caulfield, R., Wiseman, T., Gullick, J. G., & Ogilvie, R. (2022) · Journal of Clinical Nursing · doi:10.1111/jocn.16461 · Cited by 94
Lillekroken, D., Bye, A., Halvorsrud, L., Terragni, L., & Debesay, J. (2024) · Journal of Immigrant and Minority Health · doi:10.1007/s10903-023-01571-5 · Cited by 50
“The Mixed Methods Appraisal Tool (MMAT) [27] was employed as a checklist to critically evaluate the studies included in the review.”
MethodHo, T. Q. A., Le, L. K.-D., Engel, L., Le, N., Melvin, G., Le, H. N. D., & Mihalopoulos, C. (2025) · European Child & Adolescent Psychiatry · doi:10.1007/s00787-024-02386-x · Cited by 38
“As this review included empirical studies only, we decided not to use the two screening questions [34].”
Evaluating antenatal breastmilk expression outcomes: a scoping review
Foudil‐bey, I., Murphy, M. S. Q., Dunn, S., Keely, E. J., & El‐Chaâr, D. (2021) · International Breastfeeding Journal · doi:10.1186/s13006-021-00371-7 · Cited by 39
The Determinants of Infant Mortality in Brazil, 2010–2020: A Scoping Review
Bugelli, A. H., Silva, R. B. D., Dowbor, L., et al. (2021) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph18126464 · Cited by 37
“The use of this tool is suitable for many types of quantitative studies in health, as it is not only focused on randomized or case–control studies but allows the use in quantitative research also based on literature reviews and surveys [15,16].”
MethodLeighton, C., Cooper, A., Porter, A., Edwards, A., & Joseph-Williams, N. (2024) · BJGP Open · doi:10.3399/bjgpo.2023.0177 · Cited by 34
Taubner, S., Ioannou, Y., Saliba, A., Sales, C. M. D., Volkert, J., Protić, S., Adler, A., Barkauskiene, R., Conejo-Cerón, S., Di Giacomo, D., et al. (2024) · European Child & Adolescent Psychiatry · doi:10.1007/s00787-023-02186-9 · Cited by 36
“To further limit risks of bias, a formal risk of bias assessment tool (ROB-tool), the Mixed Methods Appraisal Tool [52], was used to evaluate the overall study quality.”
The relationship between theory of mind and cognition in Alzheimer’s disease: A systematic review
de Lucena, A. T., Bhalla, R. K., Santos, T. T. B. A. D., & Dourado, M. C. N. (2020) · Journal of Clinical and Experimental Neuropsychology · doi:10.1080/13803395.2019.1710112 · Cited by 42
“All included articles went through the Mixed Methods Appraisal Tool (MMAT) version 2018 (Hong et al., 2018) a critical evaluation tool that evaluates the quality of empirical studies.”
BackgroundGurtner, C., Schols, J. M. G. A., Lohrmann, C., Halfens, R. J. G., & Hahn, S. (2020) · Journal of Psychiatric and Mental Health Nursing · doi:10.1111/jpm.12712 · Cited by 42
Trauma informed care education for midwives: An integrative review
Long, T., Aggar, C., Grace, S., & Thomas, T. H. T. (2021) · Midwifery · doi:10.1016/j.midw.2021.103197 · Cited by 55
Marks, E., Moghaddam, N. G., Boos, D. D., & Malins, S. (2022) · British Journal of Health Psychology · doi:10.1111/bjhp.12628 · Cited by 47
Kassabry, M. F. (2023) · International Journal of Africa Nursing Sciences · doi:10.1016/j.ijans.2023.100553 · Cited by 49
Folder, N., Power, E., Rietdijk, R., Christensen, I., Togher, L., & Parker, D. (2023) · The Gerontologist · doi:10.1093/geront/gnad095 · Cited by 48
“Mixed-method designs were assessed using the Mixed Methods Appraisal Tool (Hong et al., 2018).”
MethodWerner, K., Alsuhaibani, S., Alsukait, R. F., Alshehri, R., Herbst, C. H., Alhajji, M., & Lin, T. K. (2023) · BMC Health Services Research · doi:10.1186/s12913-023-10059-9 · Cited by 63
“A single reviewer assessed each study for risk of bias using the Mixed Methods Appraisal Tool (MMAT) [21].”
MethodWong, J. C. M., Chua, J. Y. X., Chan, P. Y., & Shorey, S. (2024) · Journal of Advanced Nursing · doi:10.1111/jan.16127 · Cited by 83
Naser, K., Haq, Z. U., & Naughton, B. D. (2024) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph21040434 · Cited by 52
“The risk of bias was assessed using a revised Mixed-Methods Appraisal Tool (MMAT), which is a critical appraisal tool that is designed for the appraisal of systemic mixed study reviews [11].”
MethodKosiol, J., Silvester, T., Cooper, H., Alford, S., & Fraser, L. (2024) · BMC Health Services Research · doi:10.1186/s12913-024-11099-5 · Cited by 59
“We used the Mixed Methods Appraisal Tool (MMAT) 2018 [29] for quality assessment of the research reports and evidence-based articles.”
BackgroundMethodObjective Measures of Surgeon Non-Technical Skills in Surgery: A Scoping Review
Cha, J. S., & Yu, D. (2021) · Human Factors The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720821995319 · Cited by 46
“Although 11 studies were rated “Yes” for all criteria, five studies included “Can’t Tell” responses for at least one of the two screening questions, which may indicate that the paper is not an empirical study (Hong et al., 2018).”
BackgroundMethodInfluentialvan Hell‐Cromwijk, M., Metzelthin, S., Schoonhoven, L., Verstraten, C., Kroeze, W., & Ginkel, J. M. D. M. (2021) · Journal of Clinical Nursing · doi:10.1111/jocn.15747 · Cited by 46
Barriers and Enablers to Objective Testing for Asthma and COPD in Primary Care
Yamada, J., Cheung, J. L. S., Gagné, M. E., Spiegel-Feld, C., Aaron, S. D., FitzGerald, J. M., Gershon, A. S., & Gupta, S. (2021) · CHEST Journal · doi:10.1016/j.chest.2021.10.030 · Cited by 36
Polhemus, A. M., Novák, J. M., Majid, S., Simblett, S. K., Morris, D., Bruce, S., Burke, P., Dockendorf, M. F., Temesi, G., & Wykes, T. (2022) · JMIR Mental Health · doi:10.2196/25249 · Cited by 46
“Studies were critically appraised with the Mixed Methods Appraisal Tool [25,26].”
MethodThe value of social media use in improving nursing students’ engagement: A systematic review
Almutairi, M., Simpson, A., Khan, E., & Dickinson, T. (2022) · Nurse Education in Practice · doi:10.1016/j.nepr.2022.103455 · Cited by 60
Darley, A., Coughlan, B., & Furlong, E. (2021) · European Journal of Oncology Nursing · doi:10.1016/j.ejon.2021.102030 · Cited by 36
Workplace Violence Training Programs for Nursing Students: A Literature Review
Martinez, A. J. S., & Oliveira, G. C. D. (2021) · Journal of the American Psychiatric Nurses Association · doi:10.1177/1078390321994665 · Cited by 35
Badache, A. C., Hachem, H., & Mäki‐Torkko, E. (2021) · Ageing and Society · doi:10.1017/s0144686x21001070 · Cited by 44
“All the studies were quality-appraised using the Mixed Methods Appraisal Tool (MMAT) version 2018 (Hong et al. , 2018) which has been designed to assess multiple types of study designs, including mixed-methods, quantitative and qualitative studies.”
MethodInterventions to promote dementia knowledge among racial/ethnic minority groups: A systematic review
Huggins, L. K. L., Min, S. H., Dennis, C., Østbye, T., Johnson, K. S., & Xu, H. (2021) · Journal of the American Geriatrics Society · doi:10.1111/jgs.17495 · Cited by 45
“We evaluated the study quality using the Mixed Methods Appraisal Tool (MMAT).(25) This systematic review was registered in PROSPERO (CRD42020206304).”
MethodGillam, J., Davies, N., Aworinde, J., Yorganci, E., Anderson, J. E., & Evans, C. (2021) · Journal of Medical Internet Research · doi:10.2196/29837 · Cited by 40
“Where the design was not amenable to the CASP, alternative tools were used, including the Mixed Methods Appraisal Tool (MMAT) [26] and the Joanna Briggs Institute (JBI) critical appraisal tools [27].”
MethodInfluentialSeif, N. A., Bastien, R. J., Wang, B., Davies, J., Isaken, M., Ball, E., Pitman, A., & Rowe, S. L. (2022) · BJPsych Open · doi:10.1192/bjo.2021.1081 · Cited by 60
Herron, J. L., & Venner, K. L. (2023) · Journal of Racial and Ethnic Health Disparities · doi:10.1007/s40615-022-01250-5 · Cited by 35
“Quantitative coding results from the MMAT are presented in Table 1 and qualitative coding results are in Table 2.”
MethodInfluentialSports Betting Advertising: A Systematic Review of Content Analysis Studies
Killick, E. A., & Griffiths, M. D. (2022) · International Journal of Mental Health and Addiction · doi:10.1007/s11469-022-00775-4 · Cited by 60
An integrative review of the use of the concept of reassurance in clinical practice
Akyirem, S., Salifu, Y., Bayuo, J., Duodu, P. A., Bossman, I. F., & Abboah‐Offei, M. (2022) · Nursing Open · doi:10.1002/nop2.1102 · Cited by 36
Fitzpatrick, A., Wood, F., & Shepherd, V. (2022) · Trials · doi:10.1186/s13063-022-06304-x · Cited by 40
“In accordance with the MMAT guidance, the overall scores for each study were not calculated, but the rating of each criterion was presented [23].”
MethodXu, Y., Wang, Y., McCarthy, L. P., Harrison, T., & Doherty, H. (2022) · Health & Social Care in the Community · doi:10.1111/hsc.13876 · Cited by 43
Cultural diversity in health care teams: A systematic integrative review and research agenda
Schmidt, M., Steigenberger, N., Berndtzon, M., & Umans, T. (2023) · Health Care Management Review · doi:10.1097/hmr.0000000000000379 · Cited by 49
Kunnen, Y. S., Roemeling, O. P., & Smailhodžić, E. (2023) · BMC Health Services Research · doi:10.1186/s12913-023-09978-4 · Cited by 53
“Though excluding articles based on methodology is generally not recommended [23, 67], quality assessment could have provided insight into the quality of studies and enabled a sensitivity analysis [67].”
BackgroundMethodAlmukhtar, A., Batcup, C., Bowman, M., Beatty, J. W., Leff, D. R., Demirel, P., Porat, T., & Judah, G. (2023) · International Journal of Surgery · doi:10.1097/js9.0000000000000829 · Cited by 45
“Quality assessment was completed using the Mixed Methods Appraisal Tool (MMAT) [81] , due to its relevancy for both quantitative and qualitative studies.”
MethodThe stigma of smoking among women: A systematic review
David, J.-C., Fonte, D., Sutter‐Dallay, A., Auriacombe, M., Serre, F., Rascle, N., & Loyal, D. (2023) · Social Science & Medicine · doi:10.1016/j.socscimed.2023.116491 · Cited by 49
A systematic review of the nature and efficacy of Rational Emotive Behaviour Therapy interventions
King, A. M., Plateau, C. R., Turner, M. J., Young, P. A., & Barker, J. B. (2024) · PLoS ONE · doi:10.1371/journal.pone.0306835 · Cited by 60
“Methodological quality was assessed by study design in line with MMAT criteria [27].”
MethodEphraim, R. K. D., Kotam, G. P., Duah, E., Ghartey, F. N., Mathebula, E. M., & Mashamba-Thompson, T. P. (2024) · Smart Health · doi:10.1016/j.smhl.2024.100505 · Cited by 57
Wearable Devices for Supporting Chronic Disease Self-Management: Scoping Review
Gagnon, M., Ouellet, S., Attisso, E. A., Supper, W., Amil, S., Rhéaume, C., Paquette, J., Chabot, C., Laferrière, M.-C., & Sasseville, M. (2024) · Interactive Journal of Medical Research · doi:10.2196/55925 · Cited by 55
Self-citationBuragohain, D., Khichar, S., Deng, C., Meng, Y., & Chaudhary, S. (2025) · Scientific Reports · doi:10.1038/s41598-025-00916-4 · Cited by 55
“…the relevance of the selected data and further synthesizing the findings of the SLR. MMAT is used mainly for the validity and reliability of these synthesized findings while maintaining the transparency and consistency of PRISMA-based selection process of the collected and screened data 64 .”
Nursing students’ attitudes toward mobile learning: An integrative review
Chen, B., Yang, T., Wang, Y., Xiao, L., Xu, C., Shen, Y., Qin, Q., Wang, Y., Cheng, L., Chen, F., et al. (2021) · International Journal of Nursing Sciences · doi:10.1016/j.ijnss.2021.08.004 · Cited by 46
“Table 4 Quality check within MMAT.”
MethodInfluentialSuccess Factors of Medical Crowdfunding Campaigns: Systematic Review
Hou, X., Wu, T., Chen, Z., & Zhou, L. (2022) · Journal of Medical Internet Research · doi:10.2196/30189 · Cited by 44
“Papers were rated for quality using the McGill Mixed Methods Appraisal Tool version 2018 [23].”
MethodConsumer Behavior in Online-to-Offline (O2O) Commerce: A Thematic Review
Yao, P., Osman, S., Sabri, M. F., & Zainudin, N. (2022) · Sustainability · doi:10.3390/su14137842 · Cited by 81
“Suppose the quality of studies is evaluated according to the Mixed Methods Appraisal Tool (MMAT) proposed by Hong et al. [60].”
MethodIntuitive Eating Intervention and Diet Quality in Adults: A Systematic Literature Review
Hensley-Hackett, K., Bosker, J., Keefe, A. G., Reidlinger, D. P., Warner, M. M., D’Arcy, A., & Utter, J. (2022) · Journal of Nutrition Education and Behavior · doi:10.1016/j.jneb.2022.08.008 · Cited by 43
Risk Factors for Intimate Partner Violence in the Context of Disasters: A Systematic Review
Medzhitova, Y., Lai, B. S., Killenberg, P., Riobueno-Naylor, A., & Goodman, L. A. (2023) · Trauma, Violence, & Abuse · doi:10.1177/15248380221093688 · Cited by 34
“Articles were critically appraised by the two coders using an adapted version of the Mixed Methods Appraisals Tool (MMAT; Hong et al., 2018).”
MethodTransportation to school and academic outcomes: a systematic review
Hopson, L. M., Lidbe, A., Jackson, M. S., Adanu, E. K., Li, X., Penmetsa, P., Lee, H. Y., Anderson, A., Obuya, C., & Abura-Meerdink, G. (2022) · Educational Review · doi:10.1080/00131911.2022.2034748 · Cited by 45
“The MMAT is used to assess the strength of studies based on representativeness of the sample, study design, completeness of the data, and confounder management (Hong et al., 2018).”
MethodAlotni, M. A., Guilhermino, M. C., Duff, J. S., et al. (2022) · Australian Critical Care · doi:10.1016/j.aucc.2022.09.002 · Cited by 34
Baer, N.-R., Vietzke, J., & Schenk, L. (2022) · PLoS ONE · doi:10.1371/journal.pone.0278879 · Cited by 46
Reducing Excellence Gaps: A Systematic Review of Research on Equity in Advanced Education
Meyer, M. S., Shen, Y., & Plucker, J. A. (2023) · Review of Educational Research · doi:10.3102/00346543221148461 · Cited by 61
“Each article was reread and analyzed using the Mixed Methods Appraisal Tool (Hong et al., 2018), which includes two screening questions and five questions to evaluate the risk for bias in quantitative, qualitative, and mixed-methods studies.”
MethodNordin, A. M. M., Kjellström, S., Robert, G., Masterson, D., & Josefsson, K. A. (2023) · BMJ Open · doi:10.1136/bmjopen-2023-073808 · Cited by 54
Adherence to ketogenic dietary therapies in epilepsy: A systematic review of literature
Neri, L. D. C. L., Guglielmetti, M., Fiorini, S., Pasca, L., Zanaboni, M. P., GIORGIS, V. D., Tagliabue, A., & Ferraris, C. (2024) · Nutrition Research · doi:10.1016/j.nutres.2024.03.009 · Cited by 44
Gani, I., Litchfield, I. J., Shukla, D., Delanerolle, G., Cockburn, N., & Pathmanathan, A. (2025) · Journal of Medical Internet Research · doi:10.2196/62763 · Cited by 59
“The quality and bias of included papers were determined by the Critical Appraisal Skills Programme checklist for qualitative studies and the Mixed Methods Appraisal Tool for mixed method studies [55].”
Hamad, A. A., Mustaffa, D. B., Alnajjar, A. Z., Amro, R., Deameh, M. G., Amin, B., & Alkhawaldeh, I. M. (2025) · BMC Medical Education · doi:10.1186/s12909-025-07251-2 · Cited by 36
“Quality assessment was conducted by two reviewers utilizing the Mixed Methods Appraisal Tool to evaluate the various study designs (qualitative, quantitative, and mixed methods) [13].”
Body image, self-esteem, emotion regulation, and eating disorders in adults: a systematic review
Abdoli, M., Schiechtl, E., Rosato, M. S., Mangweth‐Matzek, B., Cotrufo, P., & Hüfner, K. (2025) · Neuropsychiatrie · doi:10.1007/s40211-025-00544-4 · Cited by 56
Aithal, S., Moula, Z., Karkou, V., Karaminis, T. N., Powell, J., & Makris, S. (2021) · Frontiers in Psychology · doi:10.3389/fpsyg.2021.719673 · Cited by 52
Lowther-Payne, H. J., Harrison, J., Hill, J. E., Gaskins, N. J., Green, K. L., Clegg, A. J., Connell, L., Garrett, H. M., Gibson, J. M. E., Lightbody, C. E., et al. (2021) · Implementation Science · doi:10.1186/s13012-021-01162-8 · Cited by 38
“The Mixed Methods Appraisal Tool (MMAT), a critical appraisal tool designed for reviews which include quantitative, qualitative and mixed methods studies [17], was used to assess the methodological quality of included studies.”
MethodSagen, J. S., Smedslund, G., Simonsen, A. E., Habberstad, A., Kjeken, I., Dagfinrud, H. S., & Moe, R. H. (2023) · BMJ Open Quality · doi:10.1136/bmjoq-2023-002309 · Cited by 49
“Reasons for excluding records were no full text available (n=8), wrong study design (n=8) (protocol, review or pilot), or inaccurate patient population (n=6) (inadequate consent competence, target population younger than 16 years) (figure 2).”
BackgroundMuirhead, K., Macaden, L., Smyth, K., Chandler, C., Clarke, C., Polson, R., & O’Malley, C. (2021) · Systematic Reviews · doi:10.1186/s13643-021-01781-8 · Cited by 32
“The provision of multiple methodological quality criteria for different study designs makes it most relevant for quality appraisal in mixed studies reviews [48].”
BackgroundMusculoskeletal education in undergraduate medical curricula–A systematic review
Harkins, P., Burke, E. P., & Conway, R. P. (2022) · International Journal of Rheumatic Diseases · doi:10.1111/1756-185x.14508 · Cited by 39
Nurse practitioner led telehealth services: A scoping review
Charalambous, J., Hollingdrake, O., & Currie, J. (2023) · Journal of Clinical Nursing · doi:10.1111/jocn.16898 · Cited by 42
The Wish to Hasten Death in Patients With Life-Limiting Conditions. A Systematic Overview
Rodríguez‐Prat, A., Pergolizzi, D., Crespo, I., Julià-Torras, J., Balaguer, A., Kremeike, K., Voltz, R., & Monforte‐Royo, C. (2024) · Journal of Pain and Symptom Management · doi:10.1016/j.jpainsymman.2024.04.023 · Cited by 41
Díaz, E. A., Rodríguez‐Puente, Z., Gundín, O. A., & Rivera, F. J. P. (2024) · Journal of Advanced Nursing · doi:10.1111/jan.16279 · Cited by 50
“The calculation of an overall score is discour-aged (Hong et al., 2018) but the assessment of each included study is available in Table S2.”
BackgroundMethodInfluentialA Systematic Review of Digital Competence Evaluation in Higher Education
Núñez, J. A. L., Alonso-García, S., Berral-Ortiz, B., & Victoria-Maldonado, J.-J. (2024) · Education Sciences · doi:10.3390/educsci14111181 · Cited by 128
Chen, Y., Shah, S., Chen, Y., Owen, A. J., Ekegren, C. L., Ilić, D., & Gašević, D. (2025) · BMJ Open · doi:10.1136/bmjopen-2024-095260 · Cited by 47
Metadata
- Authors
- Hong, Q. N., Fàbregues, S., Bartlett, G., Boardman, F., Cargo, M., Dagenais, P., Gagnon, M.-P., Griffiths, F., Nicolau, B., O’Cathain, A., Rousseau, M.-C., Vedel, I., & Pluye, P.
- Affiliations
- McGill University (CA)
- Universitat Oberta de Catalunya (ES)
- University of Warwick (GB)
- University of Canberra (AU)
- Université de Sherbrooke (CA)
- Université Laval (CA)
- University of Sheffield (GB)
- Institut National de la Recherche Scientifique (CA)
- Corresponding author
- Pluye, Pierre
- ORCID
- Venue
- Education for Information
- Publisher
- SAGE Publications
- Published
- 2018-12-18
- Volume
- 34
- Issue
- 4
- Pages
- 285-291
- ISSN
- 0167-8329 (ISSN-L)
- 1875-8649
- OpenAlex
- W2901724277
- Cited by
- 4,737
- References
- 21
- Citations by year
- 2026: 1312 · 2025: 974 · 2024: 800 · 2023: 610 · 2022: 501 · 2021: 370 · 2020: 143 · 2019: 22 · 2018: 3
- FWCI
- 102.21
- Type
- journal article
- Language
- en
- Open access
- green
- hdl.handle.net
- Licence
- cc-by-nc-nd
- Locations
- Education for Information (publishedVersion)
- O2 - Repositori Institucional (Universitat Oberta de Catalunya) (publishedVersion, cc-by-nc-nd)
- RECERCAT (Consorci de Serveis Universitaris de Catalunya) (publishedVersion, cc-by-nc-nd)
- White Rose Research Online (University of Leeds, The University of Sheffield, University of York) (acceptedVersion)
- Warwick Research Archive Portal (University of Warwick) (acceptedVersion)
- HAL (submittedVersion)
- eScholarship@McGill (McGill) (submittedVersion)
- LA Referencia (Red Federada de Repositorios Institucionales de Publicaciones Científicas) (submittedVersion, cc-by-nc-nd)
- eScholarship@McGill (McGill) (submittedVersion, cc-by-nc-nd)
- EspaceINRS (National Institute for Scientific Research (Canada)) (submittedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Sociology and Political Science › Delphi Technique in Research
- Social Sciences › Education › Qualitative Research Methods and Applications
- Keywords
- Mixed Methods Appraisal Tool, critical appraisal, study quality assessment, mixed methods, systematic reviews, qualitative research, quantitative research, randomized controlled trials, non-randomized studies, e-Delphi, inter-rater reliability
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.