Mixing it but not mixed-up: Mixed methods research in medical education (a critical narrative review)
Open accessThe DOI matches its Crossref record; 28 of the 28 references it cites were found in a registry or an index.
Abstract
BACKGROUND: Some important research questions in medical education and health services research need 'mixed methods research' (particularly synthesizing quantitative and qualitative findings). The approach is not new, but should be more explicitly reported. AIM: The broad search question here, of a disjointed literature, was thus: What is mixed methods research - how should it relate to medical education research?, focused on explicit acknowledgement of 'mixing'. METHODS: Literature searching focused on Web of Knowledge supplemented by other databases across disciplines. FINDINGS: Five main messages emerged: - Thinking quantitative and qualitative, not quantitative versus qualitative - Appreciating that mixed methods research blends different knowledge claims, enquiry strategies, and methods - Using a 'horses for courses' [whatever works] approach to the question, and clarifying the mix - Appreciating how medical education research competes with the 'evidence-based' movement, health services research, and the 'RCT' - Being more explicit about the role of mixed methods in medical education research, and the required expertise CONCLUSION: Mixed methods research is valuable, yet the literature relevant to medical education is fragmented and poorly indexed. The required time, effort, expertise, and techniques deserve better recognition. More write-ups should explicitly discuss the 'mixing' (particularly of findings), rather than report separate components.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
1 retracted
WITHDRAWN - Mixed Methods Sampling: A Typology With Examples
Teddlie, C., & Yu, F. (2007) · Journal of Mixed Methods Research · doi:10.1177/2345678906292430 · Cited by 2,250
RetractedMixed Methods Research: A Research Paradigm Whose Time Has Come
Johnson, R. B., & Onwuegbuzie, A. J. (2004) · Educational Researcher · doi:10.3102/0013189x033007014 · Cited by 11,170
Toward a Conceptual Framework for Mixed-Method Evaluation Designs
Greene, J. C., Caracelli, V. J., & Graham, W. F. (1989) · Educational Evaluation and Policy Analysis · doi:10.3102/01623737011003255 · Cited by 5,687
The qualitative research interview
DiCicco‐Bloom, B., & Crabtree, B. F. (2006) · Medical Education · doi:10.1111/j.1365-2929.2006.02418.x · Cited by 4,995
Gibbs, G. (2007) · doi:10.4135/9781849208574 · Cited by 4,299
Randomized, Controlled Trials, Observational Studies, and the Hierarchy of Research Designs
Concato, J., Shah, N., & Horwitz, R. I. (2000) · New England Journal of Medicine · doi:10.1056/nejm200006223422507 · Cited by 3,563 · cited 1×
“…clinical treatment areas, Benson and Hartz (2004) found that observational study evidence was reasonably solid compared with RCTs. Cohort studies are quite robust (Goldacre 2001), and Concato et al. (2004) found that allegedly inflated ‘treatment effects’ were unfounded with contemporary controls.”
Qualitative Data Analysis for Health Services Research: Developing Taxonomy, Themes, and Theory
Bradley, E. H., Curry, L. A., & Devers, K. J. (2007) · Health Services Research · doi:10.1111/j.1475-6773.2006.00684.x · Cited by 3,437 · cited 2×
“…relatively popular (Roche 1991; Bergsjø 1999) in general practice (Boulton et al. 1996; Hoddinott & Pill 1997) and health services research (Pope & Mays 1995; Shortell 1999; Hoff & Witt 2000; Bradley et al. 2007; O’Cathain et al. 2007a; Lingard et al. 2008; O’Cathain 2009; O’Cathain et al., 2009).”
BackgroundUsing Mixed-Methods Sequential Explanatory Design: From Theory to Practice
Ivankova, N. V., Creswell, J. W., & Stick, S. L. (2006) · Field Methods · doi:10.1177/1525822x05282260 · Cited by 3,167 · cited 1×
“Visually representing key ‘mixing’ decisions might help (Ivankova et al. 2006).”
BackgroundQualitative Research: Rigour and qualitative research
Mays, N., & Pope, C. (1995) · BMJ · doi:10.1136/bmj.311.6997.109 · Cited by 2,715
Checklists for improving rigour in qualitative research: a case of the tail wagging the dog?
Barbour, R. S. (2001) · BMJ · doi:10.1136/bmj.322.7294.1115 · Cited by 2,613 · cited 1×
“Competing advice about ‘doing’ qualitative research (Chapple & Rogers 1998) is off-putting, as are much criticized recipe-like checklists (Barbour 2001), and ‘feigning ‘‘immaculate e95 perception’’’ (Wolcott 1992, p. 43) (e.g. claiming exclusivity for trained social scientists).”
BackgroundA comparison of observational studies and randomized, controlled trials.
Benson, K., & Hartz, A. J. (2000) · New England Journal of Medicine · doi:10.1056/nejm200006223422506 · Cited by 2,347
Pope, C., & Mays, N. (1995) · BMJ · doi:10.1136/bmj.311.6996.42 · Cited by 2,336 · cited 1×
“…is relatively popular (Roche 1991; Bergsjø 1999) in general practice (Boulton et al. 1996; Hoddinott & Pill 1997) and health services research (Pope & Mays 1995; Shortell 1999; Hoff & Witt 2000; Bradley et al. 2007; O’Cathain et al. 2007a; Lingard et al. 2008; O’Cathain 2009; O’Cathain et…”
BackgroundThree techniques for integrating data in mixed methods studies
O'Cathain, A., Murphy, E., & Nicholl, J. (2010) · BMJ · doi:10.1136/bmj.c4587 · Cited by 1,584
Does problem-based learning work? A meta-analysis of evaluative research
Vernon, D. T., & Blake, R. L. (1993) · Academic Medicine · doi:10.1097/00001888-199307000-00015 · Cited by 1,456
Popay, J., Rogers, A., & Williams, G. (1998) · Qualitative Health Research · doi:10.1177/104973239800800305 · Cited by 987
Effectiveness of Problem-based Learning Curricula: Research and Theory
Colliver, J. A. (2000) · Academic Medicine · doi:10.1097/00001888-200003000-00017 · Cited by 917
What is Evidence-based Education?
Davies, P. (1999) · British Journal of Educational Studies · doi:10.1111/1467-8527.00106 · Cited by 650
Dixon-Woods, M., Sutton, A., Shaw, R., Miller, T., Smith, J., Young, B., Bonas, S., Booth, A., & Jones, D. (2007) · Journal of Health Services Research & Policy · doi:10.1258/135581907779497486 · Cited by 546
Problem-based learning: have the expectations been met?
Berkson, L. (1993) · Academic Medicine · doi:10.1097/00001888-199310000-00053 · Cited by 430
Appraising qualitative research articles in medicine and medical education
Côté, L., & Turgeon, J. (2005) · Medical Teacher · doi:10.1080/01421590400016308 · Cited by 312
The Case for Combining Qualitative and Quantitative Approaches in Health Services Research
Barbour, R. S. (1999) · Journal of Health Services Research & Policy · doi:10.1177/135581969900400110 · Cited by 280 · cited 9×
“qualitative research increased progressively for the last two decades, up by 26% between 2001–2005 and 2006–2010 (firstquarter) (Ring et al. 2010). ‘Mixed methods’ suffer conflicting guidance on qualitative research standards (Mays & Pope 1995; Boulton et al. 1996; Hoddinott & Pill 1997; Chapple & Rogers 1998; Popay e…”
BackgroundInfluentialAlise, M. A., & Teddlie, C. (2010) · Journal of Mixed Methods Research · doi:10.1177/1558689809360805 · Cited by 269
Moffatt, S., White, M., Mackintosh, J., & Howel, D. (2006) · BMC Health Services Research · doi:10.1186/1472-6963-6-28 · Cited by 239
Mixing Qualitative Methods: Quality Assurance or Qualitative Quagmire?
Barbour, R. S. (1998) · Qualitative Health Research · doi:10.1177/104973239800800306 · Cited by 210 · cited 2×
“Ironically, some researchers might shun mixing assumptions across the main approaches, yet happily mix methods, with very different assumptions, across traditions within qualitative research (Barbour 1998).”
BackgroundTheory in medical education research: how do we get there?
Rees, C. E., & Monrouxe, L. V. (2010) · Medical Education · doi:10.1111/j.1365-2923.2009.03615.x · Cited by 119
Critical reflections on the rise of qualitative research
Pope, C., & Mays, N. (2009) · BMJ · doi:10.1136/bmj.b3425 · Cited by 111 · cited 2×
“…research still struggles, however, for funding and mainstream medical acceptance (Cribb & Bignold 1999; Morse 2006; Dixon-Woods et al. 2007; Goguen et al. 2008; Sandelowski 2008; Pope & Mays 2009), undermined in ‘evidence’ hierarchies (Thorne 2009) and the hidden curriculum (Goguen et al. 2008).”
BackgroundQualitative research in health care: II. A structured review and evaluation of studies
Boulton, M., Fitzpatrick, R., & Swinburn, C. (1996) · Journal of Evaluation in Clinical Practice · doi:10.1111/j.1365-2753.1996.tb00041.x · Cited by 109 · cited 8×
“Beyond medical education, qualitative research is relatively popular (Roche 1991; Bergsjø 1999) in general practice (Boulton et al. 1996; Hoddinott & Pill 1997) and health services research (Pope & Mays 1995; Shortell 1999; Hoff & Witt 2000; Bradley et al. 2007; O’Cathain et al. 2007a; Lingard et…”
BackgroundInfluentialWhere is evidence‐based instructional design in medical education curriculum development?
Levinson, A. J. (2010) · Medical Education · doi:10.1111/j.1365-2923.2010.03715.x · Cited by 37 · cited 1×
“…focus more on: theorybuilding cognitive psychology (Norman 2004); illuminating the competent doctor, curriculum design (overcoming a ‘know-do’ gap (Levinson 2010)), outcomes and cost; challenging longheld assumptions that favour teaching over learning; improving both primary research and…”
Background
Cited by
Halcomb, E., & Hickman, L. (2015) · Nursing Standard · doi:10.7748/ns.29.32.41.e8858 · Cited by 465
Steinert, Y., Naismith, L., & Mann, K. (2012) · Medical Teacher · doi:10.3109/0142159x.2012.680937 · Cited by 397
Coret, A., Boyd, K., Hobbs, K., Zazulak, J., & McConnell, M. (2018) · Teaching and Learning in Medicine · doi:10.1080/10401334.2017.1398653 · Cited by 65
Kelly, M. E., Dowell, J., Husbands, A., Newell, J., O‘Flynn, S., Kropmans, T., Dunne, F. P., & Murphy, A. W. (2014) · BMC Medical Education · doi:10.1186/s12909-014-0267-0 · Cited by 50
Kondo, T., Donkers, J., Nishigori, H., Rovers, S., & Heeneman, S. (2026) · JMIR Medical Education · doi:10.2196/90064 · Cited by 1
“This study was conducted as a cross-sectional mixed methods study using a convergent design from a pragmatic perspective [37,38].”
Metadata
- Authors
- Maudsley, G.
- Venue
- Medical Teacher
- Publisher
- Informa UK Limited
- Published
- 2011-01-28
- Volume
- 33
- Issue
- 2
- Pages
- e92-e104
- ISSN
- 0142-159X (ISSN-L)
- 1466-187X
- OpenAlex
- W2122759054
- Cited by
- 99
- References
- 28
- FWCI
- 1.48
- Type
- journal article
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.