Cited by 1,107Meta-analysis and systematic reviewsDelphi Technique in ResearchHealth Policy Implementation Science

The DOI matches its Crossref record; 39 of the 40 references it cites were found in a registry or an index.

Improving the usefulness of a tool for appraising the quality of qualitative, quantitative and mixed methods studies, the Mixed Methods Appraisal Tool ( MMAT )

Open access

Hong, Quan Nha, Gonzalez‐Reyes, Araceli, Pluye, Pierre·2018·Journal of Evaluation in Clinical Practice

Open this paper in CoChat — free account Open PDF

Abstract

Rationale, aims, and objectives: Systematic reviews combining qualitative, quantitative, and/or mixed methods studies are increasingly popular because of their potential for addressing complex interventions and phenomena, specifically for assessing and improving clinical practice. A major challenge encountered with this type of review is the appraisal of the quality of individual studies given the heterogeneity of the study designs. The Mixed Methods Appraisal Tool (MMAT) was developed to help overcome this challenge. The aim of this study was to explore the usefulness of the MMAT by seeking the views and experiences of researchers who have used it. Methods: We conducted a qualitative descriptive study using semistructured interviews with MMAT users. A purposeful sample was drawn from the researchers who had previously contacted the developer of the MMAT, and those who have published a systematic review for which they had used the MMAT. All interviews were transcribed verbatim and analyzed by 2 coders using thematic analysis. Results: Twenty participants from 8 countries were interviewed. Thirteen themes were identified and grouped into the 2 dimensions of usefulness, ie, utility and usability. The themes related to utility concerned the coverage, completeness, flexibility, and other utilities of the tool. Those regarding usability were related to the learnability, efficiency, satisfaction, and errors that could be made due to difficulties understanding or selecting the items to appraise. Conclusions: On the basis of the results of this study, we make several recommendations for improving the MMAT. This will contribute to greater usefulness of the MMAT.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Integrity signals

Registry

Crossref · 10.1111/jep.12884

Cites

39 of 39 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cochrane Handbook for Systematic Reviews of Interventions: Cochrane Book Series
(2008) · doi:10.1002/9780470712184 · Cited by 25,048
Demonstrating rigor using thematic analysis: a hybrid approach of inductive and deductive coding and theme development
Fereday, J., & Muir-Cochrane, E. (2006) · International Journal of Qualitative Methods · doi:10.1177/160940690600500107 · Cited by 11,829
The integrative review: updated methodology
Whittemore, R., & Knafl, K. (2005) · Journal of Advanced Nursing · doi:10.1111/j.1365-2648.2005.03621.x · Cited by 10,241
Usability Engineering
(1993) · Elsevier eBooks · doi:10.1016/c2009-0-21512-1 · Cited by 5,552
What's in a name? Qualitative description revisited
Sandelowski, M. (2010) · Research in Nursing & Health · doi:10.1002/nur.20362 · Cited by 4,975
What is an adequate sample size? Operationalising data saturation for theory-based interview studies
Francis, J. J., Johnston, M., Robertson, C., Glidewell, L., Entwistle, V., Eccles, M. P., & Grimshaw, J. M. (2010) · Psychology & Health · doi:10.1080/08870440903194015 · Cited by 3,429
Evaluating non-randomised intervention studies
Deeks, J., Dinnes, J., D'Amico, R., Sowden, A., Sakarovitch, C., Song, F., Petticrew, M., & Altman, D. (2003) · Health Technology Assessment · doi:10.3310/hta7270 · Cited by 2,907
Qualitative description – the poor cousin of health research?
Neergaard, M. A., Olesen, F., Andersen, R. S., & Søndergaard, J. (2009) · BMC Medical Research Methodology · doi:10.1186/1471-2288-9-52 · Cited by 2,420
Content validity in psychological assessment: A functional approach to concepts and methods.
Haynes, S. N., Richard, D. C. S., & Kubany, E. S. (1995) · Psychological Assessment · doi:10.1037//1040-3590.7.3.238 · Cited by 2,387
Tools for assessing quality and susceptibility to bias in observational studies in epidemiology: a systematic review and annotated bibliography
Sanderson, S., Tatt, I. D., & Higgins, J. P. (2007) · International Journal of Epidemiology · doi:10.1093/ije/dym018 · Cited by 1,531

Flagged first, then by citations

Show all 40 references

1 of the 40 references couldn’t be matched to a known paper.

Research works printed but not linked: 5

Dixon‐Woods M, 2004, Integrative approaches to qualitative and quantitative evidence
Heyvaert M, 2016, Using Mixed Methods Research Synthesis for Literature Reviews: The Mixed Methods Research Synthesis Approach
Bai A, 2012, Quality Assessment Tools Project Report
Pluye P, Proposal: A mixed methods appraisal tool for systematic mixed studies reviews
10.1037/1040-3590.7.3.238 · DOI · Paper page

Sorted by citations

The Mixed Methods Appraisal Tool (MMAT) version 2018 for information professionals and researchers
Hong, Q. N., Fàbregues, S., Bartlett, G., Boardman, F., Cargo, M., Dagenais, P., Gagnon, M.-P., Griffiths, F., Nicolau, B., O’Cathain, A., et al. (2018) · Education for Information · doi:10.3233/efi-180221 · Cited by 4,737
“To further its development, usefulness testing was performed by interviewing 20 researchers who have used the MMAT in a systematic review or had contacted the developer of the tool for questions or permission to use the tool (Hong et al., 2018a).”
Self-citation
Improving the content validity of the mixed methods appraisal tool: a modified e-Delphi study
Hong, Q. N., Pluye, P., Fàbregues, S., Bartlett, G., Boardman, F., Cargo, M., Dagenais, P., Gagnon, M.-P., Griffiths, F., Nicolau, B., et al. (2019) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2019.03.008 · Cited by 1,082
“In addition, in interviews conducted with MMAT users to explore their views and experiences of the MMAT, concerns were raised about whether the tool included enough criteria to judge the quality of studies and criteria that were difficult to judge, in particular the criteria for qualitative and mixed methods studies […”
MethodSelf-citation
Methodological guidance for the conduct of mixed methods systematic reviews
Stern, C., Lizarondo, L. M., Carrier, J., Godfrey, C. M., Rieger, K. L., Salmond, S. W., Apóstolo, J., Kirkpatrick, P., & Loveday, H. P. (2020) · JBI Evidence Synthesis · doi:10.11124/jbisrir-d-19-00169 · Cited by 747
Synthesising quantitative and qualitative evidence to inform guidelines on complex interventions: clarifying the purposes, designs and outlining some methods
Noyes, J., Booth, A., Moore, G., Flemming, K., Tunçalp, Ö., & Shakibazadeh, E. (2019) · BMJ Global Health · doi:10.1136/bmjgh-2018-000893 · Cited by 497
Systematic review of risk and protective factors associated with substance use and abuse in individuals with autism spectrum disorders
Ressel, M., Thompson, B., Poulin, M.-H., Normand, C. L., Fisher, M. H., Couture, G., & Iarocci, G. (2020) · Autism · doi:10.1177/1362361320910963 · Cited by 394
Primary medical care continuity and patient mortality: a systematic review
Baker, R., Freeman, G. K., Haggerty, J. L., Bankart, M. J., & Nockels, K. H. (2020) · British Journal of General Practice · doi:10.3399/bjgp20x712289 · Cited by 331
Factors That Influence Participation in Physical Activity in School-Aged Children and Adolescents: A Systematic Review from the Social Ecological Model Perspective
Hu, D., Zhou, S., Crowley‐McHattan, Z. J., & Liu, Z. (2021) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph18063147 · Cited by 454
How and under what circumstances do quality improvement collaboratives lead to better outcomes? A systematic review
Zamboni, K., Baker, U., Tyagi, M., Schellenberg, J., Hill, Z. E., & Hanson, C. (2020) · Implementation Science · doi:10.1186/s13012-020-0978-z · Cited by 238
Methodological guidance for the conduct of mixed methods systematic reviews
Stern, C., Lizarondo, L. M., Carrier, J., Godfrey, C. M., Rieger, K. L., Salmond, S. W., Apóstolo, J., Kirkpatrick, P., & Loveday, H. P. (2021) · International Journal of Evidence-Based Healthcare · doi:10.1097/xeb.0000000000000282 · Cited by 214
An integrative review of leadership competencies and attributes in advanced nursing practice
Heinen, M. M., van Oostveen, C., Peters, J. W. B., Vermeulen, H., & Huis, A. M. P. (2019) · Journal of Advanced Nursing · doi:10.1111/jan.14092 · Cited by 208
Barriers and facilitators for the sustainability of digital health interventions in low and middle-income countries: A systematic review
Kaboré, S. S., Ngangue, P., Soubeiga, D., Barro, A., Pilabré, A. H., Bationo, N., Pafadnam, Y., Drabo, K. M., Hien, H., & Savadogo, L. G. B. (2022) · Frontiers in Digital Health · doi:10.3389/fdgth.2022.1014375 · Cited by 223
Child and Family Outcomes Following Pandemics: A Systematic Review and Recommendations on COVID-19 Policies
Fong, V. C., & Iarocci, G. (2020) · Journal of Pediatric Psychology · doi:10.1093/jpepsy/jsaa092 · Cited by 217
Organisational factors and non-accidental violence in sport: A systematic review
Roberts, V. L., Sojo, V. E., & Grant, F. (2019) · Sport Management Review · doi:10.1016/j.smr.2019.03.001 · Cited by 176
Women’s autonomy in healthcare decision making: a systematic review
Idris, I. B., Hamis, A. A., Bukhori, A. B. M., Hoong, D. C. C., Yusop, H., Shaharuddin, M. A.-A., Fauzi, N. A. F. A., & Kandayah, T. (2023) · BMC Women s Health · doi:10.1186/s12905-023-02792-4 · Cited by 204
Nudging toward vaccination: a systematic review
Reñosa, M. D. C., Landicho, J., Wachinger, J., Dalglish, S. L., Bärnighausen, K., Bärnighausen, T., & McMahon, S. A. (2021) · BMJ Global Health · doi:10.1136/bmjgh-2021-006237 · Cited by 170
Barriers and enablers for the implementation of trauma-informed care in healthcare settings: a systematic review
Huo, Y., Couzner, L., Windsor, T. D., Laver, K., Dissanayaka, N. N., & Cations, M. (2023) · Implementation Science Communications · doi:10.1186/s43058-023-00428-0 · Cited by 198
The Effects of Playing Video Games on Stress, Anxiety, Depression, Loneliness, and Gaming Disorder During the Early Stages of the COVID-19 Pandemic: PRISMA Systematic Review
Pallavicini, F., Pepe, A., & Mantovani, F. (2022) · Cyberpsychology Behavior and Social Networking · doi:10.1089/cyber.2021.0252 · Cited by 201
Systematic Development of Standards for Mixed Methods Reporting in Rehabilitation Health Sciences Research
Tovin, M. M., & Wormley, M. E. (2023) · Physical Therapy · doi:10.1093/ptj/pzad084 · Cited by 159
Patient feedback to improve quality of patient-centred care in public hospitals: a systematic review of the evidence
Wong, E., Mavondo, F., & Fisher, J. (2020) · BMC Health Services Research · doi:10.1186/s12913-020-05383-3 · Cited by 146
Artificial Intelligence Implementation in Healthcare: A Theory-Based Scoping Review of Barriers and Facilitators
Chomutare, T., Tejedor, M., Svenning, T. O., Marco-Ruiz, L., Tayefi, M., Lind, K., Godtliebsen, F., Moen, A., Ismail, L., Makhlysheva, A., et al. (2022) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph192316359 · Cited by 164
Rapid reviews: the pros and cons of an accelerated review process
Moons, P., Goossens, E., & Thompson, D. R. (2021) · European Journal of Cardiovascular Nursing · doi:10.1093/eurjcn/zvab041 · Cited by 158
Promoting Healthy Food Access and Nutrition in Primary Care: A Systematic Scoping Review of Food Prescription Programs
Little, M., Rosa, E., Heasley, C., Asif, A., Dodd, W., & Richter, A. (2021) · American Journal of Health Promotion · doi:10.1177/08901171211056584 · Cited by 146
A review of virtual-simulation for assessing healthcare students' clinical competency
Coyne, E., Calleja, P., Forster, E., & Lin, F. F. (2020) · Nurse Education Today · doi:10.1016/j.nedt.2020.104623 · Cited by 135
Socially assistive robots for people with dementia: Systematic review and meta-analysis of feasibility, acceptability and the effect on cognition, neuropsychiatric symptoms and quality of life
Yu, C., Sommerlad, A., Sakure, L., & Livingston, G. (2022) · Ageing Research Reviews · doi:10.1016/j.arr.2022.101633 · Cited by 131
“We used the Mixed Method Appraisal Tool (MMAT) version 2018 (Hong et al., 2018) to assess the study quality of each included study.”
Method
Mental Health during the COVID-19 Crisis in Africa: A Systematic Review and Meta-Analysis
Chen, J., Chen, J., Zhang, M., Chen, R. Z., Kechen, D. R., Dong, Z., Ye, Y., Tong, L., Chen, B. Z., Zhao, R., et al. (2021) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph182010604 · Cited by 127
Linking Quantitative and Qualitative Network Approaches: A Review of Mixed Methods Social Network Analysis in Education Research
Froehlich, D. E., Waes, S. V., & Schäfer, H. (2020) · Review of Research in Education · doi:10.3102/0091732x20903311 · Cited by 155
Factors associated with short birth interval in low- and middle-income countries: a systematic review
Pimentel, J., Ansari, U., Omer, K., Gidado, Y., Baba, M. C., Andersson, N., & Cockcroft, A. (2020) · BMC Pregnancy and Childbirth · doi:10.1186/s12884-020-2852-z · Cited by 124
What influences people’s responses to public health messages for managing risks and preventing infectious diseases? A rapid systematic review of the evidence and recommendations
Ghio, D., Lawes-Wickwar, S., Tang, M. Y., Epton, T., Howlett, N., Jenkinson, E., Stanescu, S., Westbrook, J., Kassianos, A. P., Watson, D., et al. (2021) · BMJ Open · doi:10.1136/bmjopen-2021-048750 · Cited by 125
Commercial Off-The-Shelf Video Games for Reducing Stress and Anxiety: Systematic Review
Pallavicini, F., Pepe, A., & Mantovani, F. (2021) · JMIR Mental Health · doi:10.2196/28150 · Cited by 146
Exploring the relationship between sanitation and mental and social well-being: a systematic review and qualitative synthesis
Sclar, G. D., Penakalapati, G., Caruso, B. A., Rehfuess, E. A., Garn, J. V., Alexander, K. T., Freeman, M. C., Boisson, S., Medlicott, K., & Clasen, T. (2018) · Social Science & Medicine · doi:10.1016/j.socscimed.2018.09.016 · Cited by 113
The Impact of Nurse Leadership Styles on Nurse Burnout:
Wei, H., King, A. T., Jiang, Y., Sewell, K. B., & Lake, D. (2020) · Nurse Leader · doi:10.1016/j.mnl.2020.04.002 · Cited by 112
Implementing AI in Hospitals to Achieve a Learning Health System: Systematic Review of Current Enablers and Barriers
Rahimi, A. K., Pienaar, O., Ghadimi, M., Canfell, O. J., Pole, J. D., Shrapnel, S., van der Vegt, A. H., & Sullivan, C. M. (2024) · Journal of Medical Internet Research · doi:10.2196/49655 · Cited by 139
Interprofessional Collaboration in Long-Term Care and Rehabilitation: A Systematic Review
Doornebosch, A. J., Smaling, H. J. A., & Achterberg, W. P. (2022) · Journal of the American Medical Directors Association · doi:10.1016/j.jamda.2021.12.028 · Cited by 105
Mental health literacy in children and adolescents in low- and middle-income countries: a mixed studies systematic review and narrative synthesis
Renwick, L., Pedley, R., Johnston, I., Bell, V., Lovell, K., Bee, P., & Brooks, H. L. (2022) · European Child & Adolescent Psychiatry · doi:10.1007/s00787-022-01997-6 · Cited by 121
The role of digital health technology in rural cancer care delivery: A systematic review
Morris, B. B., Rossi, B., & Fuemmeler, B. (2022) · The Journal of Rural Health · doi:10.1111/jrh.12619 · Cited by 104
A systematic review and meta-analysis of the association between shift work and metabolic syndrome: The roles of sleep, gender, and type of shift work
Khosravipour, M., Khanlari, P., Khazaie, S., Khosravipour, H., & Khazaie, H. (2021) · Sleep Medicine Reviews · doi:10.1016/j.smrv.2021.101427 · Cited by 103
Interventions to promote teamwork, delegation and communication among registered nurses and nursing assistants: An integrative review
Campbell, A. R., Layne, D. M., Scott, E. S., & Wei, H. (2020) · Journal of Nursing Management · doi:10.1111/jonm.13083 · Cited by 101
Acknowledging cancer as a family disease: A systematic review of family care in the cancer setting.
Coyne, E., Heynsbergh, N., & Dieperink, K. (2020) · European Journal of Oncology Nursing · doi:10.1016/j.ejon.2020.101841 · Cited by 90
Interventions to improve school-based eye-care services in low- and middle-income countries: a systematic review
Burnett, A. M., Yashadhana, A., Lee, L., Serova, N., Brain, D., & Naidoo, K. S. (2018) · Bulletin of the World Health Organization · doi:10.2471/blt.18.212332 · Cited by 106
Factors contributing to burnout and work-life balance in adult oncology nursing: An integrative review
Gribben, L., & Semple, C. J. (2020) · European Journal of Oncology Nursing · doi:10.1016/j.ejon.2020.101887 · Cited by 103
Barriers and enablers to the implementation of one health strategies in developing countries: a systematic review
Yopa, D. S., Massom, D. M., Kiki, G. M., Sophie, R. W., Sylvie, F., Thiam, O., Zinaba, L., & Ngangue, P. (2023) · Frontiers in Public Health · doi:10.3389/fpubh.2023.1252428 · Cited by 95
Characteristics of Case Management in Primary Care Associated With Positive Outcomes for Frequent Users of Health Care: A Systematic Review
Hudon, C., Chouinard, M., Pluye, P., Sherif, R. E., Bush, P. L., Rihoux, B., Poitras, M.-È., Lambert, M., Zomahoun, H. T. V., & Légaré, F. (2019) · The Annals of Family Medicine · doi:10.1370/afm.2419 · Cited by 99
Self-citation
Patterns and motivations of polysubstance use: a rapid review of the qualitative evidence
Boileau-Falardeau, M., Contreras, G. A., Gariépy, G., & Laprise, C. (2022) · Health Promotion and Chronic Disease Prevention in Canada · doi:10.24095/hpcdp.42.2.01 · Cited by 90
Making change last? Exploring the value of sustainability approaches in healthcare: a scoping review
Lennox, L., Linwood-Amor, A., Maher, L., & Reed, J. E. (2020) · Health Research Policy and Systems · doi:10.1186/s12961-020-00601-0 · Cited by 101
Social Media Use and Deliberate Self-Harm Among Youth: A Systematized Narrative Review
Biernesser, C., Sewall, C. J. R., Brent, D., Bear, T., Mair, C., & Trauth, J. (2020) · Children and Youth Services Review · doi:10.1016/j.childyouth.2020.105054 · Cited by 86
Barriers and facilitators to exercise adherence in community-dwelling older adults: A mixed-methods systematic review using the COM-B model and Theoretical Domains Framework
Yang, Y., Gao, Y., An, R., & Wan, Q. (2024) · International Journal of Nursing Studies · doi:10.1016/j.ijnurstu.2024.104808 · Cited by 95
Emotional response to testing positive for human papillomavirus at cervical cancer screening: a mixed method systematic review with meta-analysis
McBride, E., Tatar, O., Rosberger, Z., Rockliffe, L., Marlow, L. A. V., Moss-Morris, R. E., Kaur, N., Wade, K., & Waller, J. (2020) · Health Psychology Review · doi:10.1080/17437199.2020.1762106 · Cited by 85
Establishing a clinical service to prevent psychosis: What, how and when? Systematic review
de Pablo, G. S., Estradé, A., Cutroni, M., Andlauer, O., & Fusar‐Poli, P. (2021) · Translational Psychiatry · doi:10.1038/s41398-020-01165-x · Cited by 87
The impact of social media influencers on pregnancy, birth, and early parenting experiences: A systematic review.
Chee, R. M., Capper, T. S., & Muurlink, O. T. (2023) · Midwifery · doi:10.1016/j.midw.2023.103623 · Cited by 81
Synthesis Methods and Reporting Tool (SMART) for Research Syntheses in Applied Linguistics
Chong, S. W. (2025) · Research Synthesis in Applied Linguistics · doi:10.1080/29984475.2025.2456880 · Cited by 88
Health system readiness for non-communicable diseases at the primary care level: a systematic review
Kabir, A., Karim, M. N., Islam, R. M., Romero, L., & Billah, B. (2022) · BMJ Open · doi:10.1136/bmjopen-2021-060387 · Cited by 77
The effectiveness of smart home technologies to support the health outcomes of community-dwelling older adults living with dementia: A scoping review
Moyle, W., Murfield, J., & Lion, K. (2021) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2021.104513 · Cited by 75
Impact of the COVID-19 Pandemic on HIV Healthcare Service Engagement, Treatment Adherence, and Viral Suppression in the United States: A Systematic Literature Review
Meyer, D., Slone, S., Ogungbe, O., Duroseau, B., & Farley, J. E. (2022) · AIDS and Behavior · doi:10.1007/s10461-022-03771-w · Cited by 74
Moral distress among acute mental health nurses: A systematic review
Lamoureux, S., Mitchell, A. E., & Forster, E. (2024) · Nursing Ethics · doi:10.1177/09697330241238337 · Cited by 72
Characteristics that influence purchase choice for cannabis products: a systematic review
Donnan, J. R., Shogan, O., Bishop, L. D., Swab, M., & Najafizada, M. (2022) · Journal of Cannabis Research · doi:10.1186/s42238-022-00117-0 · Cited by 71
Healthcare professionals' dementia knowledge and attitudes towards dementia care and family carers' perceptions of dementia care in China: An integrative review
Wen-hong, Z., Jones, C., Wu, M., & Moyle, W. (2020) · Journal of Clinical Nursing · doi:10.1111/jocn.15451 · Cited by 74
A realist review to assess for whom, under what conditions and how pay for performance programmes work in low- and middle-income countries
Singh, N., Kovacs, R., Cassidy, R., Kristensen, S. R., Borghi, J., & Brown, G. W. (2020) · Social Science & Medicine · doi:10.1016/j.socscimed.2020.113624 · Cited by 77
Factors underpinning unsafe driving: A systematic literature review of car drivers
Bowen, L., Budden, S. L., & Smith, A. P. (2020) · Transportation Research Part F Traffic Psychology and Behaviour · doi:10.1016/j.trf.2020.04.008 · Cited by 68
Factors influencing clinician and patient interaction with machine learning-based risk prediction models: a systematic review
Giddings, R., Joseph, A., Callender, T., Janes, S. M., van der Schaar, M., Sheringham, J., & Navani, N. (2024) · The Lancet Digital Health · doi:10.1016/s2589-7500(23)00241-8 · Cited by 70
Spiritual assessment and spiritual care offerings as a standard of care in pediatric oncology: A recommendation informed by a systematic review of the literature
Robert, R. S., Stavinoha, P. L., Jones, B. L., Robinson, J. E., Larson, K., Hicklen, R., Smith, B. W. H., Perko, K., Koch, K. D., Findley, S., et al. (2019) · Pediatric Blood & Cancer · doi:10.1002/pbc.27764 · Cited by 63
Conceptualisations of positive mental health and wellbeing among children and adolescents in low‐ and middle‐income countries: A systematic review and narrative synthesis
Renwick, L., Pedley, R., Johnston, I., Bell, V., Lovell, K., Bee, P., & Brooks, H. L. (2021) · Health Expectations · doi:10.1111/hex.13407 · Cited by 75
The impacts of faculty caring on nursing students’ intent to graduate: A systematic literature review
Henderson, D., Sewell, K. B., & Wei, H. (2019) · International Journal of Nursing Sciences · doi:10.1016/j.ijnss.2019.12.009 · Cited by 61
The utility of physiotherapy assessments delivered by telehealth: A systematic review
Zischke, C., Simas, V., Hing, W. A., Milne, N., Spittle, A. J., & Pope, R. (2021) · Journal of Global Health · doi:10.7189/jogh.11.04072 · Cited by 68
Patient-facing digital tools for delivering genetic services: a systematic review
Lee, W., Shickh, S., Assamad, D., Luca, S., Clausen, M., Somerville, C., Tafler, A., Shaw, A., Hayeems, R. Z., & Bombard, Y. (2022) · Journal of Medical Genetics · doi:10.1136/jmg-2022-108653 · Cited by 60
The implementation of the nursing process in lower‐income countries: An integrative review
Lotfi, M., Zamanzadeh, V., Valizadeh, L., Khajehgoodari, M., Rezaei, M. E., & Khalilzad, M. A. (2019) · Nursing Open · doi:10.1002/nop2.410 · Cited by 61
Experiences of Young People and Their Caregivers of Using Technology to Manage Type 1 Diabetes Mellitus: Systematic Literature Review and Narrative Synthesis
Brew‐Sam, N., Chhabra, M., Parkinson, A., Hannan, K., Brown, E., Pedley, L., Brown, K., Wright, K., Pedley, E., Nolan, C. J., et al. (2021) · JMIR Diabetes · doi:10.2196/20973 · Cited by 65
Facilitators, barriers and considerations for the implementation of healthcare innovation: A qualitative rapid systematic review
Parmar, J., Sacrey, L. A., Anderson, S., Charles, L., Dobbs, B., McGhan, G., Shapkin, K., Tian, P., & Triscott, J. (2022) · Health & Social Care in the Community · doi:10.1111/hsc.13578 · Cited by 58
Factors Influencing Social Isolation among Cancer Patients: A Systematic Review
Wang, C., Qiu, X., Yang, X., Mao, J., & Li, Q. (2024) · Healthcare · doi:10.3390/healthcare12101042 · Cited by 67
Chatbot‐Delivered Interventions for Improving Mental Health Among Young People: A Systematic Review and Meta‐Analysis
Li, J., Li, Y., Hu, Y., Ma, C. F., Mei, X., Chan, E. A., & Yorke, J. (2025) · Worldviews on Evidence-Based Nursing · doi:10.1111/wvn.70059 · Cited by 70
Medical leadership: boon or barrier to organisational performance? A thematic synthesis of the literature
Savage, M., Savage, C., Brommels, M., & Mazzocato, P. (2020) · BMJ Open · doi:10.1136/bmjopen-2019-035542 · Cited by 57
Effects of mHealth on the psychosocial health of pregnant women and mothers: a systematic review
Sakamoto, J. L., Carandang, R. R., Kharel, M., Shibanuma, A., Львовна, Я. Е., Александровна, Б. М., & Jimba, M. (2022) · BMJ Open · doi:10.1136/bmjopen-2021-056807 · Cited by 69
Relationships and Mechanisms Between Occupational Risk Factors and Distal Upper Extremity Disorders
Keir, P. J., Farias Zuniga, A., Mulla, D. M., & Somasundram, K. G. (2021) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720819860683 · Cited by 57
Artificial intelligence and digital health in improving primary health care service delivery in LMICs: A systematic review
Saif‐Ur‐Rahman, K., Islam, M. S., Alaboson, J., Ola, O., Hasan, I., Islam, N., Mainali, S., Martina, T., Silenga, E., Muyangana, M., et al. (2023) · Journal of Evidence-Based Medicine · doi:10.1111/jebm.12547 · Cited by 56
Stigma in functional neurological disorder (FND) – A systematic review
McLoughlin, C., McWhirter, L., Pisegna, K., Tijssen, M. A. J., Tak, L. M., Carson, A. J., & Stone, J. (2024) · Clinical Psychology Review · doi:10.1016/j.cpr.2024.102460 · Cited by 64
Resilience and mental health among perinatal women: a systematic review
Hajure, M., Alemu, S. S., Abdu, Z., Mulatu, G., Alemayehu, Y., Dule, A., Hussen, M. A., Wedajo, L. F., & Gezimu, W. (2024) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2024.1373083 · Cited by 56
The Relationship Between Occupational Demands and Well-Being of Performing Artists: A Systematic Review
Willis, S., Neil, R., Mellick, M., & Wasley, D. (2019) · Frontiers in Psychology · doi:10.3389/fpsyg.2019.00393 · Cited by 69
Factors influencing the implementation of mental health recovery into services: a systematic mixed studies review
Piat, M., Wainwright, M., Sofouli, E., Vachon, B., Deslauriers, T., Préfontaine, C., & Frati, F. (2021) · Systematic Reviews · doi:10.1186/s13643-021-01646-0 · Cited by 67
Context, Input, Process, and Product Evaluation Model in medical education: A systematic review
Toosi, M., Modarres, M., Amini, M., & Geranmayeh, M. (2021) · Journal of Education and Health Promotion · doi:10.4103/jehp.jehp_1115_20 · Cited by 55
Accessibility of health care experienced by persons with dementia from ethnic minority groups and formal and informal caregivers: A scoping review of European literature
Duran, G., Uysal‐Bozkir, Ö., Uittenbroek, R. J., van Hout, H. P. J., & Groenou, M. I. B. V. (2021) · Dementia · doi:10.1177/14713012211055307 · Cited by 55
Virtual Reality Applications in Medicine During the COVID-19 Pandemic: Systematic Review
Pallavicini, F., Pepe, A., Clerici, M., & Mantovani, F. (2022) · JMIR Serious Games · doi:10.2196/35000 · Cited by 64
Use of virtual and augmented reality-based interventions in health education to improve dementia knowledge and attitudes: an integrative review
Jones, C., Jones, D., & Moro, C. (2021) · BMJ Open · doi:10.1136/bmjopen-2021-053616 · Cited by 54
A Systematic Review of Mental Health Professionals, Patients, and Carers’ Perceived Barriers and Enablers to Supporting Smoking Cessation in Mental Health Settings
Huddlestone, L., Shoesmith, E., Pervin, J., Lorencatto, F., Watson, J., & Ratschen, E. (2022) · Nicotine & Tobacco Research · doi:10.1093/ntr/ntac004 · Cited by 54
Peer support groups after acquired brain injury: a systematic review
Hughes, R. L., Fleming, P., & Henshall, L. (2020) · Brain Injury · doi:10.1080/02699052.2020.1762002 · Cited by 52
Women’s health-related vulnerabilities in natural disasters: a systematic review protocol
Fatema, S. R., Islam, M. S., East, L., & Usher, K. J. (2019) · BMJ Open · doi:10.1136/bmjopen-2019-032079 · Cited by 52
Non-pharmacological interventions to manage psychological distress in patients living with cancer: a systematic review
Paley, C. A., Boland, J. W., Santarelli, M., Murtagh, F. E., Ziegler, L. E., & Chapman, E. (2023) · BMC Palliative Care · doi:10.1186/s12904-023-01202-8 · Cited by 57
Telemedicine in the primary care of older adults: a systematic mixed studies review
Ilali, M., Berre, M. L., Vedel, I., & Khanassov, V. (2023) · BMC Family Practice · doi:10.1186/s12875-023-02085-7 · Cited by 50
Understanding barriers and facilitators to HIV testing in Canada from 2009–2019: A systematic mixed studies review
Laprise, C., & Bolster‐Foucault, C. (2021) · Canada Communicable Disease Report · doi:10.14745/ccdr.v47i02a03 · Cited by 49
The acceptability, effectiveness and gender responsiveness of participatory arts interventions in promoting mental health and Wellbeing: a systematic review
O’Donnell, S., Lohan, M., Oliffe, J. L., Grant, D., & Galway, K. (2021) · Arts & Health · doi:10.1080/17533015.2021.1894463 · Cited by 55
Nursing students' and educators' perspectives on sustainability and climate change: An integrative review
Aronsson, J., Nichols, A., Warwick, P., & Elf, M. (2023) · Journal of Advanced Nursing · doi:10.1111/jan.15950 · Cited by 49
Child and Family Outcomes and Experiences Related to Family-Centered Care Interventions for Hospitalized Pediatric Patients: A Systematic Review
Hodgson, C. R., Mehra, R., & Franck, L. S. (2024) · Children · doi:10.3390/children11080949 · Cited by 49
Determinants of Self-Care and Home-Based Management of Hypertension: An Integrative Review
Konlan, K. D., & Shin, J. (2023) · Global Heart · doi:10.5334/gh.1190 · Cited by 79
The impact of storytelling on building resilience in children: A systematic review
Ramamurthy, C., Zuo, P., Armstrong, G., & Andriessen, K. (2023) · Journal of Psychiatric and Mental Health Nursing · doi:10.1111/jpm.13008 · Cited by 78
Effect of interventions to improve safety culture on healthcare workers in hospital settings: a systematic review of the international literature
Finn, M., Walsh, A., Rafter, N., Mellon, L., Chong, H. Y., Naji, A., O'Brien, N., Williams, D. J., & McCarthy, S. E. (2024) · BMJ Open Quality · doi:10.1136/bmjoq-2023-002506 · Cited by 48
Use of academic electronic medical records in nurse education: A scoping review
Raghunathan, K., McKenna, L., & Peddle, M. (2021) · Nurse Education Today · doi:10.1016/j.nedt.2021.104889 · Cited by 47
Barriers and Facilitators Affecting the HIV Care Cascade for Migrant People Living with HIV in Organization for Economic Co-Operation and Development Countries: A Systematic Mixed Studies Review
Arora, A. K., Ortiz-Paredes, D., Engler, K., Lessard, D., Mate, K., Rodriguez-Cruz, A., Kronfli, N., Vedel, I., Cox, J., Quesnel‐Vallée, A., et al. (2021) · AIDS Patient Care and STDs · doi:10.1089/apc.2021.0079 · Cited by 46
Balancing hospital governance: A systematic review of 15 years of empirical research
Regge, M. D., & Eeckloo, K. (2020) · Social Science & Medicine · doi:10.1016/j.socscimed.2020.113252 · Cited by 53
Initiatives Addressing Precarious Employment and Its Effects on Workers’ Health and Well-Being: A Systematic Review
Gunn, V., Kreshpaj, B., Santander, N. M., Vignola, E. F., Wegman, D. H., Hogstedt, C., Ahonen, E. Q., Bodin, T., Orellana, C., Baron, S., et al. (2022) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph19042232 · Cited by 49
Partnering with frail or seriously ill patients in research: a systematic review
Ludwig, C., Graham, I. D., Gifford, W., Lavoie, J. G., & Stacey, D. (2020) · Research Involvement and Engagement · doi:10.1186/s40900-020-00225-2 · Cited by 45
A systematic literature review on designing self-regulated learning using generative artificial intelligence and its future research directions
Xia, Q., Liu, Q., Tlili, A., & Chiu, T. K. F. (2025) · Computers & Education · doi:10.1016/j.compedu.2025.105465 · Cited by 77
Digital Transformation in International Trade: Opportunities, Challenges, and Policy Implications
Mirzaye, S., & Mohiuddin, M. (2025) · Journal of Risk and Financial Management · doi:10.3390/jrfm18080421 · Cited by 86

Authors: Hong, Q. N., Gonzalez‐Reyes, A., & Pluye, P.
Affiliations: McGill University (CA)
Department of Family Medicine McGill University Montréal QC Canada
Corresponding author: Pluye, Pierre
ORCID: Hong, Quan Nha
Pluye, Pierre
Venue: Journal of Evaluation in Clinical Practice
Publisher: Wiley
Published: 2018-02-21
Volume: 24
Issue: 3
Pages: 459-467
ISSN: 1356-1294 (ISSN-L)
1365-2753
DOI: 10.1111/jep.12884
OpenAlex: W2794014169
Cited by: 1,107
References: 40
Citations by year: 2026: 227 · 2025: 230 · 2024: 176 · 2023: 156 · 2022: 130 · 2021: 78 · 2020: 84 · 2019: 20 · 2018: 5
FWCI: 34.23
Type: journal article
Language: en
Open access: green
hdl.handle.net
Licence: onlinelibrary.wiley.com/termsAndConditions#vor (vor)
Locations: Journal of Evaluation in Clinical Practice (publishedVersion)
PubMed (publishedVersion)
eScholarship@McGill (McGill) (submittedVersion)
Funders: Canadian Institutes of Health Research (301011)
Topics: Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
Social Sciences › Sociology and Political Science › Delphi Technique in Research
Health Sciences › Health Professions › General Health Professions › Health Policy Implementation Science
Keywords: Mixed Methods Appraisal Tool, mixed methods, study quality appraisal, systematic reviews, tool usability, researcher experience, complex interventions
MeSH terms: Female, Humans, Interviews as Topic, Male, Research Design, Qualitative Research, Internationality, Quality Improvement

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Improving the usefulness of a tool for appraising the quality of qualitative, quantitative and mixed methods studies, the Mixed Methods Appraisal Tool ( MMAT )

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Self-check

Citations

Cites

Cited by

Related

Metadata

Record

About these checks