The DOI matches its Crossref record; 10 of the 10 references it cites were found in a registry or an index.
Inter-Rater Reliability Methods in Qualitative Case Study Research
Open accessAbstract
The use of inter-rater reliability (IRR) methods may provide an opportunity to improve the transparency and consistency of qualitative case study data analysis in terms of the rigor of how codes and constructs have been developed from the raw data. Few articles on qualitative research methods in the literature conduct IRR assessments or neglect to report them, despite some disclosure of multiple researcher teams and coding reconciliation in the work. The article argues that the in-depth discussion and reconciliation initiated by IRR may enhance the findings and theory that emerges from qualitative case study data analysis, where the main data source is often interview transcripts or field notes. To achieve this, the article provides a missing link in the literature between data gathering and analysis by expanding an existing process model from five to six stages. The article also identifies seven factors that researchers can consider to determine the suitability of IRR to their work and it offers an IRR checklist, thereby providing a contribution to the broader literature on qualitative research methods.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Building Theories from Case Study Research
Eisenhardt, K. M. (1989) · The Academy of Management Review · doi:10.2307/258557 · Cited by 17,455
Qualitative Analysis for Social Scientists
Strauss, A. L. (1987) · doi:10.1017/cbo9780511557842 · Cited by 12,277
Fereday, J., & Muir-Cochrane, E. (2006) · International Journal of Qualitative Methods · doi:10.1177/160940690600500107 · Cited by 11,829 · cited 1×
“Interpretive rigor requires the researcher to clearly demonstrate how interpretations of the data have been realized and to illustrate fi ndings with quotations from, or access to, the raw data (Fereday and Muir-Cochrane 2006).”
Measuring nominal scale agreement among many raters.
Fleiss, J. L. (1971) · Psychological Bulletin · doi:10.1037/h0031619 · Cited by 8,733
Verification strategies for establishing reliability and validity in qualitative research
Morse, J. M., Barrett, M., Mayan, M., Olson, K., & Spiers, J. (2002) · International Journal of Qualitative Methods · doi:10.1177/160940690200100202 · Cited by 4,978
Answers to 20 questions about interrater reliability and interrater agreement
LeBreton, J. M., & Senter, J. L. (2008) · Organizational Research Methods · doi:10.1177/1094428106296642 · Cited by 3,608 · cited 1×
“Further reviews of IRR measures that build on Cohen ’ s kappa are presented, for example, by Barlow, Lai, and Azen (1991), Banerjee et al. (1999), and LeBreton and Senter (2008) for a variety of fi elds and applications.”
Content Analysis in Mass Communication: Assessment and Reporting of Intercoder Reliability
Lombard, M., Snyder-Duch, J., & Bracken, C. C. (2002) · Human Communication Research · doi:10.1111/j.1468-2958.2002.tb00826.x · Cited by 3,068
Reliability of Nominal Data Based on Qualitative Judgments
Perreault, W. D., & Leigh, L. E. (1989) · Journal of Marketing Research · doi:10.2307/3172601 · Cited by 1,575
Discovering the Future of the Case Study. Method in Evaluation Research
Yin, R. K. (1994) · Evaluation Practice · doi:10.1177/109821409401500309 · Cited by 1,039
Reliability in content analysis: some common misconceptions and recommendations.
Krippendorff, K. (2004) · Human Communication Research · doi:10.1093/hcr/30.3.411 · Cited by 1,027
Cited by
Kulkov, I., Kulkova, J., Rohrbeck, R., Menvielle, L., Kaartemo, V., & Makkonen, H. (2024) · Sustainable Development · doi:10.1002/sd.2773 · Cited by 420
Inter-Coder Agreement in Qualitative Coding: Considerations for its Use
Halpin, S. N. (2024) · American Journal of Qualitative Research · doi:10.29333/ajqr/14887 · Cited by 149
Blocksidge, H., Wittkowski, A., Heazell, A. E. P., & Smith, D. M. (2026) · Death Studies · doi:10.1080/07481187.2025.2452486 · Cited by 6
“The use of the kappa statistic is supported in qualitative health research with previous literature highlighting how using inter-rater reliability methods to assess analysis increases methodological transparency (Cole, 2024; McHugh, 2012).”
Metadata
- Authors
- Cole, R.
- Venue
- Sociological Methods & Research
- Publisher
- SAGE Publications
- Published
- 2023-02-22
- Volume
- 53
- Issue
- 4
- Pages
- 1944-1975
- ISSN
- 0049-1241 (ISSN-L)
- 1552-8294
- OpenAlex
- W4321599876
- Cited by
- 213
- References
- 10
- FWCI
- 157.90
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Inter-Rater Reliability Methods in Qualitative Case Study Research. https://app.cochat.ai/papers/10.1177/00491241231156971
- The paper
- Cole, R. (2024). Inter-Rater Reliability Methods in Qualitative Case Study Research. Sociological Methods & Research. https://doi.org/10.1177/00491241231156971
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.
