The DOI matches its Crossref record; 65 of the 65 references it cites were found in a registry or an index.
Methodological Reporting in Qualitative, Quantitative, and Mixed Methods Health Services Research Articles
Open accessAbstract
Objectives: Methodologically sound mixed methods research can improve our understanding of health services by providing a more comprehensive picture of health services than either method can alone. This study describes the frequency of mixed methods in published health services research and compares the presence of methodological components indicative of rigorous approaches across mixed methods, qualitative, and quantitative articles. Data Sources: All empirical articles (n = 1,651) published between 2003 and 2007 from four top‐ranked health services journals. Study Design: All mixed methods articles (n = 47) and random samples of qualitative and quantitative articles were evaluated to identify reporting of key components indicating rigor for each method, based on accepted standards for evaluating the quality of research reports (e.g., use of p‐values in quantitative reports, description of context in qualitative reports, and integration in mixed method reports). We used chi‐square tests to evaluate differences between article types for each component. Principal Findings: Mixed methods articles comprised 2.85 percent (n = 47) of empirical articles, quantitative articles 90.98 percent (n = 1,502), and qualitative articles 6.18 percent (n = 102). There was a statistically significant difference (χ 2(1) = 12.20, p = .0005, C ramer's V = 0.09, odds ratio = 1.49 [95% confidence interval = 1,27, 1.74]) in the proportion of quantitative methodological components present in mixed methods compared to quantitative papers (21.94 versus 47.07 percent, respectively) but no statistically significant difference (χ 2(1) = 0.02, p = .89, Cramer's V = 0.01) in the proportion of qualitative methodological components in mixed methods compared to qualitative papers (21.34 versus 25.47 percent, respectively). Conclusion: Few published health services research articles use mixed methods. The frequency of key methodological components is variable. Suggestions are provided to increase the transparency of mixed methods studies and the presence of key methodological components in published reports.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Qualitative Data Analysis: An Expanded Sourcebook
(1994) · Journal of Environmental Psychology · doi:10.1016/s0272-4944(05)80231-2 · Cited by 48,420
Lincoln, Y. S., Guba, E. G., & Pilotta, J. J. (1985) · International Journal of Intercultural Relations · doi:10.1016/0147-1767(85)90062-8 · Cited by 45,787
Research Design: Qualitative, Quantitative, and Mixed Methods Approaches
Creswell, J. W. (1994) · Cited by 35,473
(1963) · Medical Entomology and Zoology · Cited by 15,085
Research Design: Qualitative, Quantitative, and Mixed Methods Approaches
Sloane-Seale, A. (2009) · Canadian Journal of University Continuing Education · doi:10.21225/d54s3d · Cited by 14,328
Designing and Conducting Mixed Methods Research
นิลพันธุ์, ม. (2010) · Silpakorn Educational Research Journal · Cited by 9,351
Handbook of Qualitative Research.
Fine, G. A., Denzin, N., & Lincoln, Y. (1995) · Contemporary Sociology · doi:10.2307/2076550 · Cited by 7,766
(2014) · The SAGE Encyclopedia of Action Research · doi:10.4135/9781446294406.n148 · Cited by 7,250
Toward a definition of mixed methods research
Johnson, R. B., Onwuegbuzie, A. J., & Turner, L. A. (2007) · Journal of Mixed Methods Research · doi:10.1177/1558689806298224 · Cited by 6,464
Self-citationSAGE Handbook of Mixed Methods in Social & Behavioral Research
Tashakkori, A., & Teddlie, C. (2010) · doi:10.4135/9781506335193 · Cited by 6,215
Neufeld, A., & Ryan, A. G. (1988) · Canadian Journal of Program Evaluation · doi:10.3138/cjpe.3.006 · Cited by 5,913
Toward a Conceptual Framework for Mixed-Method Evaluation Designs
Greene, J. C., Caracelli, V. J., & Graham, W. F. (1989) · Educational Evaluation and Policy Analysis · doi:10.3102/01623737011003255 · Cited by 5,687
Qualitative research: standards, challenges, and guidelines
Malterud, K. (2001) · The Lancet · doi:10.1016/s0140-6736(01)05627-6 · Cited by 5,014
Messick, S. (1995) · American Psychologist · doi:10.1037/0003-066x.50.9.741 · Cited by 4,198
Teddlie, C. B., & Tashakkori, A. (2008) · Cited by 3,691
Qualitative Data Analysis for Health Services Research: Developing Taxonomy, Themes, and Theory
Bradley, E. H., Curry, L. A., & Devers, K. J. (2007) · Health Services Research · doi:10.1111/j.1475-6773.2006.00684.x · Cited by 3,437 · cited 1×
“…derived from speech or observation and attempts to understand the phenomenon of interest in terms of the meanings people bring to them (Denzin and Lincoln 1994; Shortell 1999; Giacomini and Cook for the Evidence-Based Medicine Working Group 2000; Malterud 2001; Bradley, Curry, and Devers 2007).”
BackgroundIntegrating quantitative and qualitative research: How is it done
Bryman, A. (2006) · Qualitative Research · doi:10.1177/1468794106058877 · Cited by 3,354
Hibbard, J. H., Stockard, J., Mahoney, E. R., & Tusler, M. (2004) · Health Services Research · doi:10.1111/j.1475-6773.2004.00269.x · Cited by 2,748 · cited 1×
“…development articles employed mixed methods to create measurement tools to assess, for example, caregiver burden (Cousineau et al. 2003), patient activation (Hibbard et al. 2004), and the development of a Healthcare Effectiveness Data and Information Set (HEDIS) smoking measure (Pbert et al. 2003).”
MethodQualitative Research in Health Care
(2006) · Wiley eBooks · doi:10.1002/9780470750841 · Cited by 2,712
(2008) · Cited by 2,615
Qualitative Research Design: An Interactive Approach
Pitman, M. A. (1998) · Anthropology & Education Quarterly · doi:10.1525/aeq.1998.29.4.499 · Cited by 2,405
Douglas, O. B., & Holland, B. F. (2015) · Fundamentals of educational psychology. · doi:10.1037/14910-020 · Cited by 2,272
A typology of mixed methods research designs
Leech, N. L., & Onwuegbuzie, A. J. (2009) · Quality & Quantity · doi:10.1007/s11135-007-9105-3 · Cited by 1,705
Self-citationBest Practices for Mixed Methods Research in Health Sciences: (566732013-001)
Creswell, J. W., Klassen, A. C., Clark, V. L. P., & Smith, K. C. (2011) · PsycEXTRA Dataset · doi:10.1037/e566732013-001 · Cited by 1,574
A Typology of Mixed Methods Sampling Designs in Social Science Research
Onwuegbuzie, A., & Collins, K. (2015) · The Qualitative Report · doi:10.46743/2160-3715/2007.1638 · Cited by 1,565
Self-citationFactors relevant to the validity of experiments in social settings.
Campbell, D. T. (1957) · Psychological Bulletin · doi:10.1037/h0040950 · Cited by 1,140
Pluye, P., Gagnon, M.-P., Griffiths, F., & Johnson-Lafleur, J. (2009) · International Journal of Nursing Studies · doi:10.1016/j.ijnurstu.2009.01.009 · Cited by 1,119
Qualitative and Mixed Methods Provide Unique Contributions to Outcomes Research
Curry, L. A., Nembhard, I. M., & Bradley, E. H. (2009) · Circulation · doi:10.1161/circulationaha.107.742775 · Cited by 1,085
Giacomini, M. K., Cook, D. J., & for the Evidence-Based Medicine Working Group (2000) · JAMA · doi:10.1001/jama.284.3.357 · Cited by 931
Mixing Methods: qualitative and quantitative research
(2017) · Routledge eBooks · doi:10.4324/9781315248813 · Cited by 785
Collins, K. M. T., Onwuegbuzie, A. J., & Sutton, I. L. (2006) · Learning disabilities · Cited by 771
Self-citationDefining Interdisciplinary Research: Conclusions from a Critical Review of the Literature
Aboelela, S. W., Larson, E., Bakken, S., Carrasquillo, O., Formicola, A., Glied, S. A., Haas, J., & Gebbie, K. M. (2007) · Health Services Research · doi:10.1111/j.1475-6773.2006.00621.x · Cited by 657 · cited 1×
“Health services researchers increasingly work in interdisciplinary partnerships (e.g., Aboelela et al. 2007) and use innovative methods, including mixed methods, to more fully understand health services phenomena.”
MethodCollins, K. M. T., Onwuegbuzie, A. J., & Jiao, Q. G. (2007) · Journal of Mixed Methods Research · doi:10.1177/1558689807299526 · Cited by 552 · cited 2×
“…2001 marked a turning point in the proliferation of mixed methods studies published in major electronic bibliographic databases such as PubMED (Collins, Onwuegbuzie, and Jiao 2007), we chose to examine articles from 2003 to 2007 because 2003 marks publication of the first edition of…”
BackgroundSelf-citation(2006) · Educational Researcher · doi:10.3102/0013189x035006033 · Cited by 543
Expanding the Framework of Internal and External Validity in Quantitative Research.
Onwuegbuzie, A. J. (2000) · Research in the schools · Cited by 370
Self-citationMixing Methods - Qualitative and Quantitative Research
Little, M. O., Millham, S., & Bullock, R. M. (1992) · Cited by 365
Cousineau, N., McDowell, I., Hotz, S., & Hébert, P. (2003) · Medical Care · doi:10.1097/00005650-200301000-00013 · Cited by 259
Editorial: Developing Publishable Mixed Methods Manuscripts
Creswell, J. W., & Tashakkori, A. (2007) · Journal of Mixed Methods Research · doi:10.1177/1558689806298644 · Cited by 255
Integration and Publications as Indicators of "Yield" From Mixed Methods Studies
O'Cathain, A., Murphy, E., & Nicholl, J. (2007) · Journal of Mixed Methods Research · doi:10.1177/1558689806299094 · Cited by 252
Health Services Research: An Evolving Definition of the Field
(2002) · Health Services Research · doi:10.1111/1475-6773.01020 · Cited by 237
Guidelines for Conducting and Reporting Mixed Research in the Field of Counseling and Beyond
Leech, N. L., & Onwuegbuzie, A. J. (2010) · Journal of Counseling & Development · doi:10.1002/j.1556-6678.2010.tb00151.x · Cited by 235
Self-citationThe Adoption and Practice of Mixed Methods: U.S. Trends in Federally Funded Health-Related Research
Plano Clark, V. L. (2010) · Qualitative Inquiry · doi:10.1177/1077800410364609 · Cited by 232
A Strategy to Identify Critical Appraisal Criteria for Primary Mixed-Method Studies
Sale, J. E. M., & Brazil, K. (2004) · Quality & Quantity · doi:10.1023/b:ququ.0000043126.25329.85 · Cited by 210
Improving the reporting of public health intervention research: advancing TREND and CONSORT
Armstrong, R., Waters, E., Moore, L., Riggs, E., Cuervo, L. G., Lumbiganon, P., & Hawe, P. (2008) · Journal of Public Health · doi:10.1093/pubmed/fdm082 · Cited by 171
Evaluating Mixed Research Studies: A Mixed Methods Approach
Leech, N. L., Dellinger, A. B., Brannagan, K. B., & Tanaka, H. (2010) · Journal of Mixed Methods Research · doi:10.1177/1558689809345262 · Cited by 170
Powell, H., Mihalas, S., Onwuegbuzie, A. J., Suldo, S., & Daley, C. E. (2008) · Psychology in the Schools · doi:10.1002/pits.20296 · Cited by 164
Self-citationAssessing the Quality of Mixed Methods Research: Toward a Comprehensive Framework
O'Cathain, A. (2010) · SAGE Handbook of Mixed Methods in Social & Behavioral Research · doi:10.4135/9781506335193.n21 · Cited by 161
What is driving hospitals' patient-safety efforts?
Devers, K., Pham, H. H., & Liu, G. (2004) · Health Affairs · doi:10.1377/hlthaff.23.2.103 · Cited by 136
A Framework for Reporting and Interpreting Internal Consistency Reliability Estimates
Onwuegbuzie, A. J., & Daniel, L. G. (2002) · Measurement and Evaluation in Counseling and Development · doi:10.1080/07481756.2002.12069052 · Cited by 130
Self-citationThe emergence of qualitative methods in health services research.
Shortell, S. M. (1999) · PubMed Central · Cited by 108
Climbing Up The Pay-For-Performance Learning Curve: Where Are The Early Adopters Now?
Rosenthal, M. B., Landon, B. E., Howitt, K., Song, H. R., & Epstein, A. M. (2007) · Health Affairs · doi:10.1377/hlthaff.26.6.1674 · Cited by 97
Hate Speech and Academic Freedom in the Academy
Tierney, W. G. (2006) · Educational Researcher · doi:10.3102/0013189x035003033 · Cited by 88
Qualitative Research in Health Care, 3rd Edition
Tomasic, D. M. (2007) · Medicine & Science in Sports & Exercise · doi:10.1249/mss.0b013e318031685c · Cited by 70
Leech, N. L., Onwuegbuzie, A. J., & Combs, J. P. (2011) · International Journal of Multiple Research Approaches · doi:10.5172/mra.2011.5.1.7 · Cited by 67
Self-citationA quantitative analysis of qualitative studies in clinical journals for the 2000 publishing year
McKibbon, K. A., & Gadd, C. S. (2004) · BMC Medical Informatics and Decision Making · doi:10.1186/1472-6947-4-11 · Cited by 66
Health Services Research: Work Force and Educational Issues
Field, M. J. (2013) · Cited by 65
Making Pay-For-Performance Work In Medicaid
Felt-Lisk, S., Gimm, G. W., & Peterson, S. (2007) · Health Affairs · doi:10.1377/hlthaff.26.4.w516 · Cited by 58
Consumer Enrollment and Experiences in the Cash and Counseling Program
Schore, J., Foster, L., & Phillips, B. R. (2007) · Health Services Research · doi:10.1111/j.1475-6773.2006.00679.x · Cited by 55
Perceptions of the Influence of Prescription Medicine Samples on Prescribing by Family Physicians
Hall, K. B., Tett, S. E., & Nissen, L. M. (2006) · Medical Care · doi:10.1097/01.mlr.0000204017.71426.53 · Cited by 39
Levine, C., Albert, S. M., Hokenstad, A., Halper, D., Hart, A. Y., & Gould, D. A. D. (2006) · Milbank Quarterly · doi:10.1111/j.1468-0009.2006.00449.x · Cited by 37
The John Eisenberg lecture: health services research as a citizen in improvement.
Berwick, D. M. (2005) · Europe PMC (PubMed Central) · doi:10.1111/j.1475-6773.2005.00358.x · Cited by 30
Popular Medicaid Programs Do Battle With State Budget Pressures: Perspectives From Twelve States
Hoadley, J. F., Cunningham, P. J., & McHugh, M. (2004) · Health Affairs · doi:10.1377/hlthaff.23.2.143 · Cited by 29
The John Eisenberg Lecture: Health Services Research as a Citizen in Improvement
Berwick, D. M. (2005) · Health Services Research · doi:10.1111/j.1475-6773.2005.0n359.x · Cited by 28
Determinants of HMO Formulary Adoption Decisions
Dranove, D. S., Hughes, E. F. X., & Shanley, M. T. (2003) · Health Services Research · doi:10.1111/1475-6773.t01-1-00111 · Cited by 27
Developing and Testing New Smoking Measures for the Health Plan Employer Data and Information Set
Pbert, L. A., Vuckovic, N. H., Ockene, J. K., Hollis, J. F., & Riedlinger, K. (2003) · Medical Care · doi:10.1097/01.mlr.0000053233.38183.77 · Cited by 10
Cited by
Palinkas, L. A., Horwitz, S. M., Green, C. A., Wisdom, J. P., Duan, N., & Hoagwood, K. (2015) · Administration and Policy in Mental Health and Mental Health Services Research · doi:10.1007/s10488-013-0528-y · Cited by 12,038
“First, many mixed methods studies in health services research and implementation science do not clearly identify or provide a rationale for the sampling procedure for either quantitative or qualitative components of the study (Wisdom et al. 2011), so a primary recommendation is for researchers to clearly describe thei…”
BackgroundSelf-citationLevitt, H. M., Bamberg, M., Creswell, J. W., Frost, D. M., Josselson, R., & Suárez-Orozco, C. (2018) · American Psychologist · doi:10.1037/amp0000151 · Cited by 2,286
Halcomb, E., & Hickman, L. (2015) · Nursing Standard · doi:10.7748/ns.29.32.41.e8858 · Cited by 465
Methodological Rigor in Mixed Methods: An Application in Management Studies
Harrison, R. L., Reilly, T. M., & Creswell, J. W. (2020) · Journal of Mixed Methods Research · doi:10.1177/1558689819900585 · Cited by 303
Mixed Methods in Biomedical and Health Services Research
Curry, L. A., Krumholz, H. M., O’Cathain, A., Clark, V. L. P., Cherlin, E., & Bradley, E. H. (2013) · Circulation: Cardiovascular Quality and Outcomes · doi:10.1161/circoutcomes.112.967885 · Cited by 252
Metadata
- Authors
- Wisdom, J. P., Cavaleri, M. A., Onwuegbuzie, A. J., & Green, C. A.
- Venue
- Health Services Research
- Publisher
- Wiley
- Published
- 2011-11-08
- Volume
- 47
- Issue
- 2
- Pages
- 721-745
- ISSN
- 0017-9124 (ISSN-L)
- 1475-6773
- OpenAlex
- W1973928213
- Cited by
- 299
- References
- 65
- FWCI
- 19.64
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.