Toward an Agenda for Evaluation of Qualitative Research
The DOI matches its Crossref record; 78 of the 79 references it cites were found in a registry or an index.
Abstract
Evaluation is essential for research quality and development, but the diversity of traditions that characterize qualitative research suggests that general checklists or shared criteria for evaluation are problematic. We propose an approach to research evaluation that encourages reflexive dialogue through use of an evaluation agenda . In proposing an evaluation agenda we shift attention from rule-based judgment to reflexive dialogue. Unlike criteria, an agenda may embrace pluralism, and does not request consensus on ontological, epistemological, and methodological issues, only consensus on what themes warrant discussion. We suggest an evaluation agenda—EPICURE—with two dimensions communicated through use of two acronyms.The first, EPIC, refers to the challenge of producing rich and substantive accounts based on engagement, processing, interpretation, and (self-)critique. The second—CURE—refers to the challenge of dealing with preconditions and consequences of research, with a focus on (social) critique, usefulness, relevance, and ethics. The seven items of the composite agenda EPICURE are presented and exemplified. Features and implications of the agenda approach to research evaluation are then discussed.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Some authors have raised concerns about the use of such checklists and adherence to strict, universal criteria because they do not afford sufficient flexibility to accommodate the diverse approaches and multiple interpretive practices often represented in qualitative studies.[7][8][9][10][11]”
From A Review of the Quality Indicators of Rigor in Qualitative Research (2020) · cites this paper as [8]
In the citing paper
“Participants' perceptions of ethical conduct are fundamental to a relationship likely to generate high quality data.”
From A Review of the Quality Indicators of Rigor in Qualitative Research (2020) · cites this paper as [8]
Citations
Cites
The Structure of Scientific Revolutions
Kuhn, T. S., & Schlegel, R. (1963) · Physics Today · doi:10.1063/1.3050879 · Cited by 35,538
(1984) · Cited by 21,166
Situated Knowledges: The Science Question in Feminism and the Privilege of Partial Perspective
Haraway, D. (1988) · Feminist Studies · doi:10.2307/3178066 · Cited by 18,364
The Interpretation of Cultures
(2017) · Macat Library eBooks · doi:10.4324/9781912128310 · Cited by 16,449
Qualitative Analysis for Social Scientists
Strauss, A. L. (1987) · doi:10.1017/cbo9780511557842 · Cited by 12,277
Ambrose, A., Wittgenstein, L., & Anscombe, G. E. M. (1954) · Philosophy and Phenomenological Research · doi:10.2307/2103745 · Cited by 12,192
Gadamer, H. G., Weinsheimer, J. C., & Marshall, D. G. (1960) · Cited by 9,337
THE STRUCTURE OF SCIENTIFIC REVOLUTIONS
MCGUIRE, J. E. (1963) · Philosophical Books · doi:10.1111/j.1468-0149.1963.tb00813.x · Cited by 8,424
The New production of knowledge: the dynamics of science and research in contemporary societies
(1995) · Choice Reviews Online · doi:10.5860/choice.32-4463 · Cited by 7,046
Situated Knowledges: The Science Question in Feminism and the Privilege of Partial Perspective
Haraway, D. (1988) · PhilPapers (PhilPapers Foundation) · Cited by 6,992
Paradigmatic Controversies, Contradictions, and Emerging Confluences.
Guba, E. G., & Lincoln, Y. S. (2005) · Cited by 6,808
The Sage Handbook of Qualitative Research[1]
(2006) · Qualitative Research in Organizations and Management: An International Journal · doi:10.1108/17465640610666642 · Cited by 5,842
Criteria for assessing the trustworthiness of naturalistic inquiries
Guba, E. G. (1981) · ECTJ · doi:10.1007/bf02766777 · Cited by 5,527
Qualitative research: standards, challenges, and guidelines
Malterud, K. (2001) · The Lancet · doi:10.1016/s0140-6736(01)05627-6 · Cited by 5,014
Self-citationVerification strategies for establishing reliability and validity in qualitative research
Morse, J. M., Barrett, M., Mayan, M., Olson, K., & Spiers, J. (2002) · International Journal of Qualitative Methods · doi:10.1177/160940690200100202 · Cited by 4,978
The New Production of Knowledge: The Dynamics of Science and Research in Contemporary Societies.
Baber, Z., Gibbons, M., Limoges, C., Nowotny, H., Schwartzman, S., Scott, P., & Trow, M. (1995) · Contemporary Sociology · doi:10.2307/2076669 · Cited by 4,406
Reflexive Methodology: New Vistas for Qualitative Research
(2012) · QMiP Bulletin · doi:10.53841/bpsqmip.2012.1.14.101 · Cited by 3,463
Checklists for improving rigour in qualitative research: a case of the tail wagging the dog?
Barbour, R. S. (2001) · BMJ · doi:10.1136/bmj.322.7294.1115 · Cited by 2,613 · cited 1×
“Barbour (2001) has argued that although checklists might have contributed to acceptance of qualitative methods in medicine, they can be counterproductive if used prescriptively.”
BackgroundEvolving guidelines for publication of qualitative research studies in psychology and related fields
Elliott, R., Fischer, C. T., & Rennie, D. L. (1999) · British Journal of Clinical Psychology · doi:10.1348/014466599162782 · Cited by 2,491
Constructivist, interpretivist approaches to human inquiry.
Schwandt, T. A. (1994) · Cited by 2,476
Dennes, W. R., & Dewey, J. (1940) · The Philosophical Review · doi:10.2307/2180803 · Cited by 2,470
Dilemmas in qualitative health research
Yardley, L. (2000) · Psychology & Health · doi:10.1080/08870440008400302 · Cited by 2,310 · cited 1×
“…2000; Angen, 2000; Koro-Ljungberg, 2008), and therefore requires a flexible framework instead of fixed criteria (see Cohen & Crabtree, 2008 Yardley, 2000, 2007).4 For instance, Aigen (1996) argued that standards developed for evaluation should not be considered as rules to be followed,…”
BackgroundThe Normative Structure of Science
Merton, R. Κ. (1973) · Medical Entomology and Zoology · Cited by 2,265
The constitution of society: Outline of the theory of structuration
Palm, R. (1986) · Political Geography Quarterly · doi:10.1016/0260-9827(86)90040-6 · Cited by 1,590
The Cambridge dictionary of philosophy
(1996) · Choice Reviews Online · doi:10.5860/choice.33-3059 · Cited by 1,182
Carter, S. M., & Little, M. (2007) · Qualitative Health Research · doi:10.1177/1049732307306927 · Cited by 1,141
The therapy of desire: theory and practice in Hellenistic ethics
(1995) · Choice Reviews Online · doi:10.5860/choice.32-2664 · Cited by 1,133
Beyond objectivism and relativism
Bernstein, R. J. (1983) · Cited by 1,096
Beyond Objectivism and Relativism: Science, Hermeneutics, and Praxis.
Larmore, C., & Bernstein, R. J. (1985) · The Philosophical Review · doi:10.2307/2185249 · Cited by 1,084
ETHICS AND POLITICS IN QUALITATIVE RESEARCH
Christians, C. G. (2005) · Cited by 1,079
Evaluating Interpretive Inquiry: Reviewing the Validity Debate and Opening the Dialogue
Angen, M. J. (2000) · Qualitative Health Research · doi:10.1177/104973200129118516 · Cited by 998
Post-Modernism and the Social Sciences: Insights, Inroads, and Intrusions
Rosenau, P. M. (1992) · Princeton University Press eBooks · doi:10.1515/9781400820610 · Cited by 989
Popay, J., Rogers, A., & Williams, G. (1998) · Qualitative Health Research · doi:10.1177/104973239800800305 · Cited by 987
Reflexive methodology - new vistas for qualitative research
Draper, P. (2001) · Journal of Advanced Nursing · doi:10.1046/j.1365-2648.2001.0435d.x · Cited by 952
(2003) · Digital Education Resource Archive (University College London) · Cited by 902
Evaluative Criteria for Qualitative Research in Health Care: Controversies and Recommendations
Cohen, D. J., & Crabtree, B. F. (2008) · The Annals of Family Medicine · doi:10.1370/afm.818 · Cited by 829
Steier, F. (1991) · Digital Commons - University of South Florida (University of South Florida) · Cited by 804
Post-modernism and the social sciences: insights, inroads, and intrusions
(1992) · Choice Reviews Online · doi:10.5860/choice.30-0617 · Cited by 803
The Therapy of Desire: Theory and Practice in Hellenistic Ethics
Nussbaum, M. C. (2009) · Princeton University Press eBooks · doi:10.1515/9781400831944 · Cited by 788
Real Science: What it Is and What it Means
Ziman, J. (2000) · doi:10.1017/cbo9780511541391 · Cited by 760
Postmodernism and the Social Sciences
(1992) · Palgrave Macmillan UK eBooks · doi:10.1007/978-1-349-22183-7 · Cited by 748
Reflexivity : a practical guide for researchers in health and social sciences
Finlay, L., & Gough, B. (2003) · Cited by 705
Werner, A., & Malterud, K. (2003) · Social Science & Medicine · doi:10.1016/s0277-9536(02)00520-8 · Cited by 696
Self-citationDemonstrating Validity in Qualitative Psychology
Yardley, L. (2024) · Qualitative Psychology: A Practical Guide to Research Methods · doi:10.4135/9781036232764.n12 · Cited by 658
Reflexivity: A Practical Guide for Researchers in Health and Social Sciences
(2003) · Wiley eBooks · doi:10.1002/9780470776094 · Cited by 554
Meyrick, J. (2006) · Journal of Health Psychology · doi:10.1177/1359105306066643 · Cited by 402
Sandelowski, M., & Barroso, J. (2002) · International Journal of Qualitative Methods · doi:10.1177/160940690200100107 · Cited by 335
Culture-Centered Music Therapy
Stige, B. (2015) · Oxford Handbooks Online · doi:10.1093/oxfordhb/9780199639755.013.1 · Cited by 303
Self-citationShared understanding of the qualitative research process. Guidelines for the medical researcher
MALTERUD, K. (1993) · Family Practice · doi:10.1093/fampra/10.2.201 · Cited by 278
Self-citationReflexivity: A Practical Guide for Researchers in Health and Social Sciences
Finlay, F. (2008) · Cited by 177
Dewey, J. (1931) · Medical Entomology and Zoology · Cited by 169
Morse, J. M. (2006) · Qualitative Health Research · doi:10.1177/1049732305285482 · Cited by 156
Johansson, A. W., & Lindhult, E. (2008) · Action Research · doi:10.1177/1476750307083713 · Cited by 138
Validity and Validation in the Making in the Context of Qualitative Research
Koro‐Ljungberg, M. (2008) · Qualitative Health Research · doi:10.1177/1049732308318039 · Cited by 120
A Review Committee's Guide for Evaluating Qualitative Proposals
Morse, J. M. (2003) · Qualitative Health Research · doi:10.1177/1049732303013006005 · Cited by 99
Instances or Sequences? Improving the State of the Art of Qualitative Research
Silverman, D. J. (2008) · Forum qualitative Sozialforschung · doi:10.17169/fqs-6.3.6 · Cited by 89
Principles of qualitative research
Aigen, K. S. (1995) · Cited by 77
Birch, S., & Gafni, A. (2004) · Routledge eBooks · doi:10.4324/9780203493694 · Cited by 75
Hvas, L. (2001) · Maturitas · doi:10.1016/s0378-5122(01)00184-0 · Cited by 64
University of California Publications in Philosophy.
Picard, M., Adams, G. P., Loewenberg, J., & Pepper, S. C. (1926) · The Journal of Philosophy · doi:10.2307/2013815 · Cited by 59
Action research and curriculum development
McKernan, J. (1987) · Peabody Journal of Education · doi:10.1080/01619568709538548 · Cited by 50
Standards of Integrity for Qualitative Music Therapy Research
Bruscia, K. E. (1998) · Journal of Music Therapy · doi:10.1093/jmt/35.3.176 · Cited by 50
Being a psychiatric patient in the community- reclassified as the stigmatized "other' '
Thesen, J. (2001) · Scandinavian Journal of Public Health · doi:10.1177/14034948010290040901 · Cited by 47
Individual Music Therapy with Persons with Frontotemporal Dementia: Singing Dialogue
Ridder, H. M., & Aldridge, D. (2005) · Nordic Journal of Music Therapy · doi:10.1080/08098130509478132 · Cited by 47
Steihaug, S., Ahlsen, B., & Kirsti, M. (2001) · Scandinavian Journal of Primary Health Care · doi:10.1080/02813430152706783 · Cited by 44
On the Idea of What Constitutes Good Qualitative Inquiry
Engel, J. D., & Kuzel, A. J. (1992) · Qualitative Health Research · doi:10.1177/104973239200200409 · Cited by 41
Multiple Perspectives: A Guide to Qualitative Research in Music Therapy
Smeijsters, H. (1998) · Cited by 34
From a medical consultation to a written text:2. Pragmatics and textlinguistics applied to medicine
Nessa, J. (1995) · Scandinavian Journal of Primary Health Care · doi:10.3109/02813439508996742 · Cited by 24
Playin' in the band : a qualitative study of popular music styles as clinical improvisation
Aigen, K. S. (2005) · Cited by 21
The Argumentative Quality of the Qualitative Research Report
Smaling, A. (2002) · International Journal of Qualitative Methods · doi:10.1177/160940690200100304 · Cited by 17
Some Pragmatic Thoughts About Evaluating Qualitative Health Research
Kuzel, A. J., & Engel, J. D. (2001) · Sage eBooks · doi:10.4135/9781412986236.n6 · Cited by 17
From a medical consultation to a written text:1. Transcribing the doctor-patient dialogue
Nessa, J. (1995) · Scandinavian Journal of Primary Health Care · doi:10.3109/02813439508996741 · Cited by 16
Quantitative Influences on the Presentation of Qualitative Articles
Morse, J. M. (2007) · Qualitative Health Research · doi:10.1177/1049732306297321 · Cited by 14
Playin’ in the Band: A Qualitative Study of Popular Music Styles as Clinical Improvisation
Kenny, C. B. (2004) · The Arts in Psychotherapy · doi:10.1016/j.aip.2004.01.003 · Cited by 14
The role of values in qualitative music therapy research
Aigen, K. S. (1996) · Cited by 11
An investigation of meaning in clinical music improvisation with troubled adolescents
Gardstrom, S. C. (2019) · doi:10.25335/306c-2147 · Cited by 10
Mitcham, C. (2007) · Qualitative Health Research · doi:10.1177/1049732307308951 · Cited by 8
Feminism, Epistemology and Morality
Holst, C. (2010) · Cited by 5
Research works printed but not linked: 15
- Abrams, B. ( 2005). Evaluating qualitative music therapy research. In B. Wheeler (Ed.), Music therapy research (2nd ed.; pp. 246-258). Gilsum, NH: Barcelona.
- 10.1177/104973230001000308 · doi:10.1177/104973230001000308 · Paper page
- Blaxter, M., 1996, Medical Sociology News, 22, 68
- Bourdieu, P., 2007, Viten om viten og refleksivitet [Science of science and reflexivity]
- Freire, P., 1968, De undertryktes pædagogik [Pedagogy of the oppressed. Original title: Pedagogia do oprimido] (S. G. Borgen, Trans., 1993)
- Habermas, J., 1968, Erkenntnis und Interesse. Mit einem neuen Nachwort [Knowledge and human interests, with a new afterword]
- Jensen, U.J., 2002, Medicin som humaniora [Medicine as humanity]
- Konstan, D., 2005, Epicurus. The Stanford encyclopedia of philosophy
- Kvale, S., 1996, InterViews. An introduction to qualitative research writing
- Sedley, D.N., 1995, Epicureanism
- Silverman, D., 2005, Forum: Qualitative Social Research, 6, 30
- Skjervheim, H., 1959, Objektivismen-og studiet av mennesket [Objectivism-and the study of man.]
- Smaling, A., 2002, Retrieved
- Smith, J.K., 2000, The problem of criteria in the age of relativism
- Thesen, J., 2001, Scandinavian Journal of Public Health, 29, 248
Cited by
Standards for Reporting Qualitative Research: A Synthesis of Recommendations
O’Brien, B. C., Harris, I. B., Beckman, T. J., Reed, D. A., & Cook, D. A. (2014) · Academic Medicine · doi:10.1097/acm.0000000000000388 · Cited by 12,211
Systematic text condensation: A strategy for qualitative analysis
Malterud, K. (2012) · Scandinavian Journal of Public Health · doi:10.1177/1403494812465030 · Cited by 2,305
“Synthesizing the contents of the condensates, we develop descriptions and concepts, providing credible stories that can make a difference by elucidating the study question [2].”
BackgroundSelf-citationA Review of the Quality Indicators of Rigor in Qualitative Research
Johnson, J. L., Adkins, D., & Chauvin, S. (2020) · American Journal of Pharmaceutical Education · doi:10.5688/ajpe7120 · Cited by 2,076
“Researchers must be respectful of the participants’ contributions and quotes, and results must be reported truthfully and honestly.(8) Interview methods range from highly structured to increase dependability or completely open-ended to allow for interviewers to clarify a participant’s response for increased credibilit…”
BackgroundHannes, K., Lockwood, C., & Pearson, A. (2010) · Qualitative Health Research · doi:10.1177/1049732310378656 · Cited by 409
Gütekriterien qualitativer Forschung
Flick, U. (2010) · Handbuch Qualitative Forschung in der Psychologie · doi:10.1007/978-3-531-92052-8_28 · Cited by 367
Metadata
- Authors
- Stige, B., Malterud, K., & Midtgarden, T.
- Affiliations
- University of Bergen (NO)
- Uni Research (Norway) (NO)
- Corresponding author
- Stige, Brynjulf
- Venue
- Qualitative Health Research
- Publisher
- SAGE Publications
- Published
- 2009-10
- Volume
- 19
- Issue
- 10
- Pages
- 1504-1516
- ISSN
- 1049-7323 (ISSN-L)
- 1552-7557
- OpenAlex
- W2104762235
- Semantic Scholar
- 2c41149a5df3b9c8293825c2f7e4afb45e864679
- Cited by
- 327
- References
- 79
- Citations by year
- 2026: 15 · 2025: 19 · 2024: 15 · 2023: 28 · 2022: 28 · 2021: 17 · 2020: 32 · 2019: 16 · 2018: 21 · 2017: 19 · 2016: 17 · 2015: 21 · 2014: 22 · 2013: 22 · 2012: 10
- FWCI
- 47.77
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- journals.sagepub.com/page/policies/text-and-data-mining-license (tdm)
- Locations
- Qualitative Health Research (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Education › Qualitative Research Methods and Applications
- Social Sciences › Sociology and Political Science › Qualitative Research Methods and Ethics
- Social Sciences › Public Administration › Evaluation and Performance Assessment
- Keywords
- research evaluation, qualitative research, EPICURE, reflexive dialogue, research quality, research ethics, research relevance, social criticism, pluralism
- MeSH terms
- Communication, Data Collection, Data Interpretation, Statistical, Humans, Research Design, Knowledge, Qualitative Research, Checklist
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.