The DOI matches its Crossref record; 50 of the 50 references it cites were found in a registry or an index.
Improving the content validity of the mixed methods appraisal tool: a modified e-Delphi study
Open accessAbstract
OBJECTIVE: The mixed methods appraisal tool (MMAT) was developed for critically appraising different study designs. This study aimed to improve the content validity of three of the five categories of studies in the MMAT by identifying relevant methodological criteria for appraising the quality of qualitative, survey, and mixed methods studies. STUDY DESIGN AND SETTING: First, we performed a literature review to identify critical appraisal tools and extract methodological criteria. Second, we conducted a two-round modified e-Delphi technique. We asked three method-specific panels of experts to rate the relevance of each criterion on a five-point Likert scale. RESULTS: A total of 383 criteria were extracted from 18 critical appraisal tools and a literature review on the quality of mixed methods studies, and 60 were retained. In the first and second rounds of the e-Delphi, 73 and 56 experts participated, respectively. Consensus was reached for six qualitative criteria, eight survey criteria, and seven mixed methods criteria. These results led to modifications of eight of the 11 MMAT (version 2011) criteria. Specifically, we reformulated two criteria, replaced four, and removed two. Moreover, we added six new criteria. CONCLUSION: Results of this study led to improve the content validity of this tool, revise it, and propose a new version (MMAT version 2018).
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“As a result, this review employed Hong et al.’s [75] Mixed Methods Appraisal Tool (MMAT), as the instrument provides reviewers with a single tool to assess quantitative, qualitative and mixed methods research, and has shown improved content validity over recent years [76].”
From The Mental Health of Elite-Level Coaches: A Systematic Scoping Review (2024) · cites this paper as [76]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The Cochrane Collaboration's tool for assessing risk of bias in randomised trials
Higgins, J. P. T., Altman, D. G., Gotzsche, P. C., Juni, P., Moher, D., Oxman, A. D., Savovic, J., Schulz, K. F., Weeks, L., Sterne, J. A. C., et al. (2011) · BMJ · doi:10.1136/bmj.d5928 · Cited by 33,817
Cochrane Handbook for Systematic Reviews of Interventions: Cochrane Book Series
(2008) · doi:10.1002/9780470712184 · Cited by 25,048
GRADE guidelines: 1. Introduction—GRADE evidence profiles and summary of findings tables
Guyatt, G., Oxman, A. D., Akl, E. A., Kunz, R., Vist, G., Brozek, J., Norris, S., Falck-Ytter, Y., Glasziou, P., & deBeer, H. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2010.04.026 · Cited by 10,681 · cited 1×
“Several approaches for rating the overall quality of evidence have been developed, such as Grading of Recommendations, Assessment, Development and Evaluations (GRADE) [38] and GRADE-Confidence in the Evidence from Reviews of Qualitative research (CERQual) [39].”
MethodAchieving integration in mixed methods designs-principles and practices
Fetters, M. D., Curry, L. A., & Creswell, J. W. (2013) · Health Services Research · doi:10.1111/1475-6773.12117 · Cited by 7,219 · cited 2×
“Also, Fetters, Curry, and Creswell [31] identified three integration levels (design, methods, and interpretation/reporting).”
BackgroundDetermination and Quantification Of Content Validity
LYNN, M. R. (1986) · Nursing Research · doi:10.1097/00006199-198611000-00017 · Cited by 6,775 · cited 2×
“78 or higher were indicative of good content validity [48].”
BackgroundResearch guidelines for the Delphi survey technique
Hasson, F. (2000) · Journal of Advanced Nursing · doi:10.1046/j.1365-2648.2000.01567.x · Cited by 5,951 · cited 1×
“The Delphi technique is used to reach consensus among a group of experts [20] and is particularly suitable to build consensus on issues that have limited or contradictory evidence [21].”
MethodIs the CVI an acceptable indicator of content validity? Appraisal and recommendations
Polit, D. F., Beck, C. T., & Owen, S. V. (2007) · Research in Nursing & Health · doi:10.1002/nur.20199 · Cited by 5,273 · cited 1×
“Polit, Beck, and Owen [49] recommended having 8 to 12 experts for the first round.”
BackgroundPrimary, Secondary, and Meta-Analysis of Research
GLASS, G. V. (1976) · Educational Researcher · doi:10.3102/0013189x005010003 · Cited by 4,890 · cited 1×
“Initial developments mainly focused on meta-analysis for addressing questions on the effectiveness of interventions, and the emphasis was on randomized controlled trials [2,3].”
MethodSystematic reviews in the social sciences: a practical guide
(2006) · Choice Reviews Online · doi:10.5860/choice.43-5664 · Cited by 4,081
The Handbook of Research Synthesis.
Howel, D., Cooper, H., & Hedges, L. V. (1995) · Biometrics · doi:10.2307/2533295 · Cited by 3,657
Cited by
Sorted by citations
Items 601–700 of 1,001
Caponnetto, P., Triscari, S., Prezzavento, G. C., Farruggio, G., Farrauto, C., Lanzafame, S., Schilirò, G., Uccelli, E., Vitale, N. M., Fakhrou, A. A., et al. (2024) · Health Psychology Research · doi:10.52965/001c.120958 · Cited by 3
Mukurunge, E., Nyoni, C. N., & Hugo, L. (2024) · Nurse Education in Practice · doi:10.1016/j.nepr.2024.104240 · Cited by 3
González-Devesa, D., Ayán, C., Sánchez-Lastra, M. A., Gutiérrez-Hong, C., García-Fresneda, A., & Diz, J. C. (2024) · Revista de Neurología · doi:10.31083/rn36478 · Cited by 3
“The methodological quality of comparative and single arms studies were assessed by means of the Mixed Meth-ods Appraisal Tool (MMAT) [14].”
Dissolving contradictory demands: A systematic review of alcohol use of working mothers
Patsouras, M., Wright, C. C. J., Caluzzi, G., Kuntsche, E. N., & Kuntsche, S. (2025) · Drug and Alcohol Dependence · doi:10.1016/j.drugalcdep.2025.112664 · Cited by 3
A Systematic Review of Educators' Overreliance and Misuse of AI in Student Feedback
Hulus, A. (2025) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-3373-0122-8.ch015 · Cited by 3
Kunorubwe, T. (2025) · The Cognitive Behaviour Therapist · doi:10.1017/s1754470x25100263 · Cited by 4
Milne‐Ives, M., Neill, S., Bayes, N., Blair, M. E., Blewitt, J., Bray, L., Carrol, E. D., Carter, B., Dawson, R., Dimitri, P., et al. (2021) · JMIR Research Protocols · doi:10.2196/27504 · Cited by 4
“They will be measured using the Mixed-Methods Appraisal Tool (MMAT) [34].”
MethodEstévez‐López, F., Levelt, L., Veer, J. V. ‘., Hrehovcsik, M. M., Visch, V., Bramer, W. M., Hillegers, M. H. J., van de Putte, E. M., Veltkamp, R. C., & Nijhof, S. L. (2022) · BMJ Open · doi:10.1136/bmjopen-2021-059581 · Cited by 4
Virtual supervision in ophthalmology: a scoping review
Kang, C., Shin, C. J., Scott, I. U., & Greenberg, P. B. (2023) · Graefe s Archive for Clinical and Experimental Ophthalmology · doi:10.1007/s00417-023-06048-7 · Cited by 3
Berube, M., Verret, M., Martorella, G., Gagnon, M., Bourque, L., Déry, M.-P., Hudon, A., Singer, L. N., Richard‐Denis, A., Ouellet, S., et al. (2023) · JBI Evidence Synthesis · doi:10.11124/jbies-22-00298 · Cited by 3
Self-citationCollins, T. L., Cardella, A., & Gordon, S. (2023) · Home Healthcare Now · doi:10.1097/nhh.0000000000001187 · Cited by 3
Geraghty, T. J., Foster, M. M., & Burridge, L. H. (2023) · Journal of Continuing Education in the Health Professions · doi:10.1097/ceh.0000000000000521 · Cited by 3
Eckersley, R., & Kosiol, J. (2023) · Asia Pacific Journal of Health Management · doi:10.24083/apjhm.v18i3.2473 · Cited by 2
“To ensure the quality of included articles a quality assessment was conducted using the Mixed Methods Appraisal Tool (MMAT), 2018 version 17 [25].”
MethodBungay, H., Walshe, N., & Dadswell, A. (2024) · Cogent Education · doi:10.1080/2331186x.2023.2298047 · Cited by 3
“In addition, following Hong et al. (2019), an agreement index was calculated, where for each of the implementation outcomes (AIM, IAM, and FIM) the number of participants rating criteria as 4 or 5 (agree or strongly agree) and whose total score for the dimension was 16 or above was divided by the…”
Minian, N., Wong, M., Hafuth, S., Rodak, T., Rahimi, A., Gjomema, D., Rose, J., Zawertailo, L. A., Ratto, M., & Selby, P. L. (2024) · BMC Public Health · doi:10.1186/s12889-024-18139-z · Cited by 4
“…Critical Appraisal Tools [28] to assess the quality of randomized controlled trials (RCTs), quasi-experimental, analytical cross-sectional, case–control, cohort, and qualitative studies, and the Mixed Methods Appraisal Tool (MMAT) version 2018 [29] to assess the quality of mixed methods studies.”
The Impact of Social Vulnerability on Substance Use Detection Practices in Pregnancy
Lijewski, V. A., Aldrich, H., & Straub, H. L. (2024) · American Journal of Perinatology · doi:10.1055/s-0044-1782686 · Cited by 3
Consensus on the definition and attributes of person‐centered teamwork: An e‐Delphi study
Viljoen, A., Leech, R., Slater, P. F., & Heyns, T. (2024) · Worldviews on Evidence-Based Nursing · doi:10.1111/wvn.12724 · Cited by 3
Process Evaluations of Diabetes Self-Management Programs: A Systematic Review of the Literature
Nsobundu, C., Nmadu, Y. W., Wagle, N. S., Foster, M., McKyer, E. L. J., Sherman, L. D., Ory, M. G., & Burdine, J. N. (2024) · American Journal of Health Promotion · doi:10.1177/08901171241238554 · Cited by 3
Consumer perspectives on antibiotic use in residential aged care: A mixed-methods systematic review
Gyawali, R., Gamboa, S., Rolfe, K., Westbrook, J. I., & Raban, M. Z. (2024) · American Journal of Infection Control · doi:10.1016/j.ajic.2024.07.008 · Cited by 3
“29 Any disagreements in scoring were discussed to reach a consensus, with a third reviewer (MR) available to resolve disputes.”
Valarezo-Bravo, O. F., Guzmán‐González, M., & Garrido-Rojas, L. (2024) · Revista Latinoamericana de Psicología · doi:10.14349/rlp.2024.v56.11 · Cited by 3
A systematic review and meta-analysis focusing on the use of ayurvedic medicine in cerebral palsy
Rocha, A. M. D. O., Len, C. A., Santos, F. H., Iglesias, S., & Bissoto, J. R. (2024) · Journal of Ayurveda and Integrative Medicine · doi:10.1016/j.jaim.2024.101035 · Cited by 3
“The quality of the selected studies was assessed using MMAT (Mixed Methods Appraisal Tool) in its 2018 [20] version and GRADE, (Grading of Recommendations, Assessment, Development and Evaluations) tools, it was considered the lowest [18].”
MethodAuditory hallucinations simulation in mental health nursing education: a scoping review
Parozzi, M., Bozzetti, M., Ferrara, P., Mancin, S., Destrebecq, A. L. L., Sguanci, M. E., Gazzelloni, A., Fantuzzi, C., Lusignani, M., & Terzoni, S. (2024) · Teaching and learning in nursing · doi:10.1016/j.teln.2024.10.004 · Cited by 3
Khodke, K. S., & Vijay, N. R. (2025) · Cureus · doi:10.7759/cureus.77760 · Cited by 3
Children's and adolescents’ descriptors of asthma symptoms: An integrative review
Wallace‐Farquharson, T., Rhee, H., Duckworth, L. J., Elder, J. H., & Wilkie, D. J. (2022) · International Journal of Nursing Studies Advances · doi:10.1016/j.ijnsa.2022.100063 · Cited by 3
A Scoping Review on Nutrition Challenges Among People Living With HIV/AIDS in Sub-Saharan Africa
Dzinamarira, T., Pierre, G., Chitungo, I., Habtu, M., & Okova, R. (2019) · Global Journal of Health Science · doi:10.5539/gjhs.v11n12p109 · Cited by 3
“The 2018 version of the Mixed Methods Appraisal Tool (MMAT) was used for the evaluation of the risk of bias for the studies, which were included in the review (Hong et al., 2019).”
MethodChemane, N., Cobbing, S., & Chetty, V. (2020) · JMIR Research Protocols · doi:10.2196/19039 · Cited by 3
Joberns, L. (2021) · British Journal of Cardiac Nursing · doi:10.12968/bjca.2021.0076 · Cited by 3
Dwyer, L., Dowding, D., & Kearney, R. (2022) · BMJ Open · doi:10.1136/bmjopen-2021-055587 · Cited by 3
Nursing activities for health promotion in palliative home care: an integrative review
Leclerc-Loiselle, J., Gendron, S., & Daneault, S. (2024) · Palliative Care and Social Practice · doi:10.1177/26323524241235191 · Cited by 5
Non-ictal, interictal and ictal déjà vu: a systematic review and meta-analysis
Hadzic, A., & Andersson, S. (2024) · Frontiers in Neurology · doi:10.3389/fneur.2024.1406889 · Cited by 2
Nasir, B., Vinayagam, R., Massi, L., Nagaraj, S. H., Toombs, M. R., & Rae, K. M. (2024) · Molecular Genetics & Genomic Medicine · doi:10.1002/mgg3.70018 · Cited by 2
“The MMAT has shown to have high validity (Hong et al. 2019) and reliability (Souto et al. 2015), and is a useful tool for appraising literature studies (Hong, Gonzalez-Reyes, and Pluye 2018).”
MethodDisposal of Unused Antibiotics in Community Pharmacies in Saudi Arabia: A Mixed-Methods Study
Aloudah, N. M., Alsaja, R., Mohareb, A. B., Alshabanah, A., & Alammari, R. (2024) · Integrated Pharmacy Research and Practice · doi:10.2147/iprp.s488482 · Cited by 2
Newington, L., Ceh, D., Sandford, F. M., Parsons, V., & Madan, I. (2024) · Journal of Occupational Rehabilitation · doi:10.1007/s10926-024-10251-6 · Cited by 3
“Additionally, the mixed meth-ods critical appraisal tool [22] was used to appraise mixed methods studies.”
Iwuchukwu, G., Anttila, M., Lantta, T., Varpula, J., & Ameel, M. (2025) · CIN Computers Informatics Nursing · doi:10.1097/cin.0000000000001263 · Cited by 3
Vaccarezza, S., Labbé, N., López, A., Mundt, A. P., & Errázuriz, P. (2025) · Trauma Violence & Abuse · doi:10.1177/15248380251335040 · Cited by 2
Aguirre, A. O., Soliman, M. A. R., Stockman, I. G., Boojraj, G. R., Quiceno, E., Khan, A., Rho, K., Pollina, J., & Mullin, J. P. (2025) · European Spine Journal · doi:10.1007/s00586-025-09051-0 · Cited by 2
Herakal, G. I., Challangod, P. K., Kandakurti, P. K., & Shetty, P. (2025) · Cureus · doi:10.7759/cureus.87143 · Cited by 2
Boersema, G. C., Botma, Y., & Mulder, M. (2025) · International Journal of Older People Nursing · doi:10.1111/opn.70044 · Cited by 2
“Consensus required an agreement index of ≥ 0.8, aligning with rigorous standards cited in literature (Hong et al. 2019; Nasa et al. 2021; Waggoner et al. 2016).”
InfluentialAlmubarak, A., Alshibani, A. N., & Walker, S. (2025) · Birth · doi:10.1111/birt.70014 · Cited by 2
Kelly, S. B., Pryor, C. A., Stephens, M., & Heaslip, V. (2025) · Journal of Advanced Nursing · doi:10.1111/jan.70212 · Cited by 2
“The Mixed Methods Appraisal Tool (MMAT) (Hong et al. 2019) was used to assess all empirical research.”
Systematic Literature Review: Indoor Lighting and Color Effects on Persons With ASD
Zaikina, V., Thorud, H.-M. S., Rustad, S. F., & Falkenberg, H. K. (2025) · HERD Health Environments Research & Design Journal · doi:10.1177/19375867251373096 · Cited by 4
“…conducted using three standardized checklists, for example, JBI Critical Appraisal Checklist for Analytical Cross-Sectional Studies (Moola et al., 2017), JBI Critical Appraisal Checklist for Qualitative Research (Lockwood et al., 2015), and Mixed Methods Appraisal Tool (MMAT) (Hong et al., 2019).”
Budman, J. R., Maeir, A., Bar-Ilan, R. T., & Golos, A. (2025) · Occupational Therapy In Health Care · doi:10.1080/07380577.2025.2589258 · Cited by 2
Yang, M., Liang, J., Zhang, L., Liu, H., Chen, Y., Wang, Y., Qi, C., Ma, Y., Gao, Z., Zhang, X., et al. (2025) · Clinical and Translational Allergy · doi:10.1002/clt2.70113 · Cited by 3
First-time fathers’ inclusion in, and engagement with, perinatal primary care: An integrative review
Richard, P., Kalembo, F. W., Kuliukas, L. J., Kothari, A., Kendall, G. E., Priest, R., & Steen, M. (2025) · Midwifery · doi:10.1016/j.midw.2025.104685 · Cited by 2
Patient Participation During Nursing Bedside Handover: A State-of-the-Art Review
Cruchinho, P. J. M., Teixeira, G. M., Lucas, P. R. M. B., Gaspar, F., & López-Franco, M. D. (2025) · Nursing Reports · doi:10.3390/nursrep15120438 · Cited by 2
Rajaofera, M. J. N., Liu, W., Reziky, A. M., Tomboanona, S., Kuang, D., & Xia, Q. (2026) · Malaria Journal · doi:10.1186/s12936-025-05718-7 · Cited by 2
Adolescents' experiences of living with type 2 diabetes mellitus: A scoping review
Osei, E. P., Antwi, E. O., Ekpor, E., Osei, G. Y., Asante, A., Dzotrah, G. E., & Boakye, M. D. S. (2026) · Journal of Pediatric Nursing · doi:10.1016/j.pedn.2026.01.002 · Cited by 2
Prasetia, A. R., Muchamad, W., Noraga, G. B., Suryadi, E., & Rasto (2026) · Novatio Journal of Management Technology and Innovation · doi:10.61978/novatio.v4i1.1259 · Cited by 2
Sacchetti, A., Berube, M., Pison, C. M., Gagnon, M., & Bellier, A. (2026) · European Journal of General Practice · doi:10.1080/13814788.2026.2613495 · Cited by 3
Self-citationA systematic review to guide measles exposure periods for contact tracing
Wellham, H. F., Gillani, S., Ameaka, F., Campbell, E. A., Toner, E. S., Ravi, S., Soneja, S. I., & Rivers, C. (2026) · EBioMedicine · doi:10.1016/j.ebiom.2026.106138 · Cited by 5
Maas, A., Bertolini, E., Ostheim, K., Lie, H. C., Demoor-Goldschmidt, C., Garami, M., Michel, G., & Ilic, A. (2026) · Patient Education and Counseling · doi:10.1016/j.pec.2026.109506 · Cited by 2
Islam, M. S., Jafari, A., Bhattacharya, A., & Zhu, Y. (2026) · Energy and Buildings · doi:10.1016/j.enbuild.2026.117241 · Cited by 2
Ai-based interventions for adolescent mental health: a systematic review
Saputra, R., Hayati, I. R., Lidyawati, Y., Oktasari, M., & Noviza, N. (2026) · Journal of Behavioral and Cognitive Therapy · doi:10.1016/j.jbct.2026.100595 · Cited by 2
Campos-Beltrán, D., Zhang, S., & Marshall, L. (2026) · Frontiers in Sleep · doi:10.3389/frsle.2026.1802882 · Cited by 3
Pétrin, R., Beauchamp, K., Perreault, A., & Forest, G. (2026) · Sleep Medicine · doi:10.1016/j.sleep.2026.108966 · Cited by 2
The social legacy potential of the Paralympic Games: a rapid review of the evidence
Brown, C., & Kersting, F. (2026) · International Journal of Sport Policy and Politics · doi:10.1080/19406940.2026.2662209 · Cited by 2
“If a study used quantitative or mixed methods, the Mixed Methods Appraisal Tool (MMAT) was operationalised (Hong et al .”
Abagnale, M., Palazzo, C., Zampetti, N., Filippo, M. D., Citarella, R., Abagnale, F. G., Cecere, L., Limonti, F., & Gravante, F. (2026) · Healthcare · doi:10.3390/healthcare14101286 · Cited by 2
“For mixed-methods study designs, methodological quality was assessed using the Mixed Methods Appraisal Tool (MMAT, 2018 version) [39].”
Martinez-Navarro, O., & Martínez‐Martínez, C. (2026) · Journal of Public Health · doi:10.1007/s10389-026-02797-x · Cited by 2
Vaccine Hesitancy in Autism Spectrum Disorder: A Systematic Review With Meta-Analyses
Gülle, B. T., Doğruyol, A. R., & Ören, M. M. (2026) · Journal of Autism and Developmental Disorders · doi:10.1007/s10803-026-07456-4 · Cited by 2
Yuan, Y., Deng, M., Chen, H., Lu, X., Chen, X., Bai, D., Gao, H., Hou, C., & Gao, J. (2026) · Therapeutic Advances in Endocrinology and Metabolism · doi:10.1177/20420188261469945 · Cited by 2
Lukacs, M. J., Kowalski, K. L., Peters, N., Stanley, M., & Rushton, A. B. (2022) · BMJ Open · doi:10.1136/bmjopen-2022-061475 · Cited by 2
Budiarti, R., Siswantoyo, S., Sukamti, E. R., Sriwahyuniati, C. F., Iswanto, A., Latif, R. A., Sutoro, S., & Miftachurochmah, Y. (2022) · International journal of human movement and sports sciences · doi:10.13189/saj.2022.100503 · Cited by 2
Tavares, S., & Morini, S. (2023) · British Journal of Cardiac Nursing · doi:10.12968/bjca.2022.0086 · Cited by 2
Evaluation of HIV screening in hospital emergency services. Systematic review
Valero-Verdejo, L., Hueso‐Montoro, C., & Pérez‐Morente, M. Á. (2023) · International Emergency Nursing · doi:10.1016/j.ienj.2023.101355 · Cited by 2
SOLEDAD Y FRAGILIDAD EN PERSONAS MAYORES QUE VIVEN EN COMUNIDAD: UNA REVISIÓN INTEGRATIVA
Díaz, B. C., & Araya, A.-X. (2023) · Horizonte de enfermeria · doi:10.7764/horiz_enferm.num.esp.332-348 · Cited by 2
How to write a “results section” in biomedical scientific research papers?: Critical review
Habtu, Y., & Deressa, W. A. (2024) · Research Methods in Medicine & Health Sciences · doi:10.1177/26320843241237444 · Cited by 2
A Comprehensive Guide to Conducting Systematic Reviews
Cong‐Lem, N. (2024) · Advances in educational technologies and instructional design book series · doi:10.4018/979-8-3693-6482-6.ch006 · Cited by 2
A review of factors influencing the uptake of prostate cancer treatment in Nigeria
IHEANACHO, C. O., & Odili, V. U. (2024) · Journal of Cancer Policy · doi:10.1016/j.jcpo.2024.100487 · Cited by 2
Suryawanshi, S. V., Dwidmuthe, K. S., Savalkar, S., & Bhalerao, A. V. (2024) · Cureus · doi:10.7759/cureus.67365 · Cited by 2
“For the included studies, the Mixed Methods Appraisal Tool (MMAT) was used for the assessment of methodological quality to evaluate studies that are qualitative, quantitative descriptive (cross-sectional), non-randomized, randomized controlled trials, and mixed methods [22,23].”
MethodDerboghossian, G., Foust, J. B., & Hayman, L. L. (2024) · Journal of Religion and Health · doi:10.1007/s10943-024-02147-9 · Cited by 3
Chester, H., Bradbury, B., Santer, M., Morrison, L. G., Fader, M. J., Ward, J., Manthorpe, J., & Murphy, C. L. (2024) · BMC Geriatrics · doi:10.1186/s12877-024-05465-5 · Cited by 2
How to Encourage the Public to Help People With Dementia in the Community: A Scoping Review
Zhi, S., Li, R., Fang, S., Gao, S., Song, D., Wu, Y., Zhong, Q., Jin, C., Zhang, Y., Song, H., et al. (2024) · Journal of Advanced Nursing · doi:10.1111/jan.16698 · Cited by 2
The burden and care of orofacial cleft in Rwanda: a scoping review
Amzat, J., Murererehe, J., Uwambaye, P., Yusuf, L., Olaopa, O., Salami, A. A., Harerimana, I. E., Ineza, M. C., Ntigura, E., Ober-Oluoch, J., et al. (2025) · BMC Oral Health · doi:10.1186/s12903-025-05900-y · Cited by 3
“The data extraction sheet was developed through a review of existing scoping reviews [21, 22] and robust discussions among team members.”
Jaam, M. J., Awaisu, A., Stewart, D. C., Mukhalalati, B., Sethi, A., Elshazly, M., Abdelrahman, A., & Hadi, M. A. (2025) · PLoS ONE · doi:10.1371/journal.pone.0321587 · Cited by 2
“…with Diverse Studies), MMR-RHS (Standards for Mixed Methods Reporting in Rehabilitation Health Sciences Research), ASSESS tool (a comprehenSive tool to Support rEporting and critical appraiSal of qualitative, quantitative, and mixed methods implementation reSearch outcomes) [22,25,34–38].”
Ekongefeyin, S. N. N., Dedino, G. A., & Wild, C. E. K. (2025) · Heart · doi:10.1136/heartjnl-2025-325992 · Cited by 4
The role, challenges, and impact of occupational therapists in custodial settings: A scoping review
Elder, E. G., Werner, S., & Crilly, J. L. (2025) · Australian Occupational Therapy Journal · doi:10.1111/1440-1630.70042 · Cited by 1
“The remaining MMAT considerations were then applied to the included publications according to the study design (Hong et al., 2019).”
InfluentialHagen, M. H., Jaccheri, L., Papavlasopoulou, S., Hartvigsen, G., & Sæter, T. (2025) · Systematic Reviews · doi:10.1186/s13643-025-03028-2 · Cited by 1
Adzordor, P., Avoka, C., Bawontuo, V., Agbesi, S., & Kuupiel, D. (2020) · Systematic Reviews · doi:10.1186/s13643-020-01546-9 · Cited by 2
Yoshimura, M., & Sumi, N. (2022) · Journal of International Nursing Research · doi:10.53044/jinr.2021-0001 · Cited by 2
The relational dynamics of women with obstetric fistula in Africa: a narrative review
Karega, M. (2020) · African Journal of Midwifery and Women s Health · doi:10.12968/ajmw.2019.0018 · Cited by 2
Dunnack, H., Judge, M. P., Cong, X. S., Salner, A., Duffy, V. B., & Xu, W. (2021) · Oncology nursing forum · doi:10.1188/21.onf.317-331 · Cited by 2
Brunet, J., Price, J., & Baillot, A. (2021) · BMJ Open · doi:10.1136/bmjopen-2021-057309 · Cited by 2
“Following recommendations, 24 appraisal will be discussed qualitatively and quantitative quality scores will not be derived.”
MethodClinimetric study and review of the Reaction Level Scale
Anestis, D. M., Monioudis, P. M., Foroglou, N., Tsonidis, C. A., & Tsitsopoulos, P. P. (2022) · Acta Neurologica Scandinavica · doi:10.1111/ane.13604 · Cited by 2
Infecciones del tracto urinario en mujeres embarazadas mexicanas: una revisión sistemática
Ortiz, M. I., Corona-Olivera, E. J., Cariño‐Cortés, R., & Fernández‐Martínez, E. (2022) · Educación y Salud Boletín Científico Instituto de Ciencias de la Salud Universidad Autónoma del Estado de Hidalgo · doi:10.29057/icsa.v10i20.8560 · Cited by 2
Parr, J., Chen, Y.-F., Damery, S., & Grove, A. (2022) · NIHR Open Research · doi:10.3310/nihropenres.13307.2 · Cited by 3
An introduction to carrying out and appraising systematic reviews in healthcare
Davies, A. (2023) · Journal of Paramedic Practice · doi:10.12968/jpar.2023.15.11.cpd1 · Cited by 3
Adebisi, R. O. (2022) · Journal of Learning and Educational Policy · doi:10.55529/jlep.31.1.12 · Cited by 2
Xu, J., Lio, A., Dhaliwal, H., Andrei, S.-T., Balakrishnan, S., Nagani, U., & Samadder, S. (2021) · Research Square · doi:10.21203/rs.3.rs-607400/v1
“[31] Hong QN, Pluye P, Fàbregues S, Bartlett G, Boardman F, Cargo M, et al.”
BackgroundMethodEditorial: Applying Transformative Approaches in Mixed Methods Research in Psychology
Shannon-Baker, P., Perez, A., McCrudden, M. T., & Fàbregues, S. (2025) · Methods in Psychology · doi:10.1016/j.metip.2025.100222 · Cited by 2
Self-citationJagoda, T., & Rathnayake, S. (2022) · International Health Trends and Perspectives · doi:10.32920/ihtp.v2i2.1649 · Cited by 1
Gartner, J., Benharbit, B., Layani, G., Sasseville, M., Lemaire, C., Bergeron, F., Wilhelmy, C., Menear, M., & Côté, A. (2023) · BMJ Open · doi:10.1136/bmjopen-2023-073767 · Cited by 1
“67”
MethodAljumah, L. A., & Alhaddad, F. A. (2023) · Journal of International Special Needs Education · doi:10.9782/jisne-d-22-00008 · Cited by 1
Physical Violence and Aggression in Parkinson's Disease: A Systematic Review
Jones, M. B., Gibson, L., Gimenez-Zapiola, M., Guerra, A., Bhatti, G. S., Broadway, D., Tea, J. C., Prasad, A., Gates, R., Hinton, E. G., et al. (2024) · Journal of the Academy of Consultation-Liaison Psychiatry · doi:10.1016/j.jaclp.2024.01.007 · Cited by 2
Learner-System Interaction: Examining Learners’ E-Learning Experience in Distance Education
Nyirongo, D. K., & Mbano, N. (2024) · Open Journal of Social Sciences · doi:10.4236/jss.2024.1210043 · Cited by 1
Long, S., Madon, Z., Norowi, N. M., & Ang, M. F. (2025) · Frontiers in Public Health · doi:10.3389/fpubh.2024.1431996 · Cited by 1
“The MMAT was used as it is a more reliable and valid quality assessment tool for critical evaluation of qualitative, quantitative, and mixed-method studies (30).”
Freeman, L., Smith, A., Ku, N., Ayala, A. P., Bennie, F., Berger, A. S., Chesney, T. R., Desforges, P., Haase, K. R., Jones, J. S., et al. (2025) · PLoS ONE · doi:10.1371/journal.pone.0317177 · Cited by 1
Xiang, Y., Ding, R., Yuan, B., Wu, J., Lu, Y., & Yang, X. (2025) · Systematic Reviews · doi:10.1186/s13643-025-02788-1 · Cited by 1
“Two reviewers will evaluate the methodological rigor of selected studies using the 2018 version of the Mixed Methods Assessment Tool (MMAT) [30].”
Cyclical medication management interventions in health care settings: A systematic review
Meulenbroeks, I., Mercado, C., Urwin, R., Seaman, K. L., Kelly, A., Qadri, O., & Westbrook, J. I. (2025) · Learning Health Systems · doi:10.1002/lrh2.70005 · Cited by 1
von der Lühe, V., Roos, M., Löbberding, M., Peter, S., Scholten, N., Köpke, S., & Dichter, M. (2025) · Journal of Advanced Nursing · doi:10.1111/jan.17053 · Cited by 1
Gains, A. M., Moghaddam, N. G., Baliousis, M., & Biswas, S. (2025) · BMJ Supportive & Palliative Care · doi:10.1136/spcare-2024-005342 · Cited by 2
“32 The rating system for this tool is the following: ‘Yes’, ‘No’ or ‘Can’t tell’ in reference to if the paper reports appropriate details.”
Metadata
- Authors
- Hong, Q. N., Pluye, P., Fàbregues, S., Bartlett, G., Boardman, F., Cargo, M., Dagenais, P., Gagnon, M.-P., Griffiths, F., Nicolau, B., O’Cathain, A., Rousseau, M.-C., & Vedel, I.
- Affiliations
- McGill University (CA)
- Universitat Oberta de Catalunya (ES)
- University of Warwick (GB)
- University of Canberra (AU)
- Université de Sherbrooke (CA)
- Université Laval (CA)
- University of Sheffield (GB)
- Institut National de la Recherche Scientifique (CA)
- Corresponding author
- Pluye, Pierre
- ORCID
- Venue
- Journal of Clinical Epidemiology
- Publisher
- Elsevier BV
- Published
- 2019-07
- Volume
- 111
- Pages
- 49-59.e1
- ISSN
- 0895-4356 (ISSN-L)
- OpenAlex
- W2924322406
- Cited by
- 1,082
- References
- 50
- Citations by year
- 2026: 205 · 2025: 178 · 2024: 190 · 2023: 174 · 2022: 163 · 2021: 110 · 2020: 57 · 2019: 5
- FWCI
- 168.24
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Journal of Clinical Epidemiology (publishedVersion, cc-by-nc-nd)
- PubMed (publishedVersion)
- The Oberta in Open Access (Universitat Oberta de Catalunya) (publishedVersion, cc-by-nc-nd)
- White Rose Research Online (University of Leeds, The University of Sheffield, University of York) (acceptedVersion)
- HAL (submittedVersion, cc-by-nc-nd)
- eScholarship@McGill (McGill) (submittedVersion)
- EspaceINRS (National Institute for Scientific Research (Canada)) (submittedVersion)
- RECERCAT (Consorci de Serveis Universitaris de Catalunya) (submittedVersion, cc-by-nc-nd)
- Warwick Research Archive Portal (University of Warwick) (submittedVersion)
- Topics
- Social Sciences › Sociology and Political Science › Delphi Technique in Research
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Health Sciences › Nursing › Research and Theory › Health Education and Validation
- Keywords
- Mixed Methods Appraisal Tool, content validity, critical appraisal, methodological quality criteria, qualitative research, survey research, mixed methods, e-Delphi, expert consensus
- MeSH terms
- Systematic Reviews as Topic, Delphi Technique, Research Design, Reproducibility of Results
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
