Cited by 1,027Reliability and Agreement in MeasurementComputational and Text Analysis MethodsPsychometric Methodologies and Testing

The DOI matches its Crossref record; 5 of the 5 references it cites were found in a registry or an index.

Reliability in content analysis: some common misconceptions and recommendations.

Open access

Krippendorff, K.·2004·Human Communication Research

DOI 10.1093/hcr/30.3.411

Open this paper in CoChat — free account Open PDF

Abstract

In a recent article published in this journal, Lombard, Snyder-Duch, and Bracken (2002) surveyed 200 content analyses for their reporting of reliability tests; compared the virtues and drawbacks of five popular reliability measures; and proposed guidelines and standards for their use. Their discussion revealed that numerous misconceptions circulate in the content analysis literature regarding how these measures behave and can aid or deceive content analysts in their effort to ensure the reliability of their data. This paper proposes three conditions for statistical measures to serve as indices of the reliability of data and examines the mathematical structure and the behavior of the five coefficients discussed by the authors, plus two others. It compares common beliefs about these coefficients with what they actually do and concludes with alternative recommendations for testing reliability in content analysis and similar data-making efforts.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Integrity signals

Registry

Crossref · 10.1093/hcr/30.3.411

Cites

0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

The publisher hasn’t deposited this paper’s reference list.

Research works printed but not linked: 13

Bennett, E. M., Alpert, R., & Goldstein, A. C. (1954). Communications through limited response questioning. Public Opinion Quarterly, 18, 303-308.
Craig, R. T. (1981). Generalization of Scott"s index of intercoder agreement. Public Opinion Quarterly, 45, 260-264.
Goodman, L. A, & Kruskal, W. H. (1954). Measures of association for cross classifications. Journal of the American Statistical Association, 49, 732-764.
Holsti, O. R. (1969). Content analysis for the social sciences and humanities. Reading, MA: Addison- Wesley.
Hughes, M. A., & Garrett, D. E. (1990). Intercoder reliability estimation -Approaches in marketing: A generalizability theory framework for quantitative data. Journal of Marketing Research, 27, 185- 195.
Krippendorff, K. (1970). Bivariate agreement coefficients for reliability data. In E. R. Borgatta & G. W. Bohrnstedt (Eds.), Sociological methodology 1970 (pp. 139-150). San Francisco, CA: Jossey Bass.
Krippendorff, K. (2004). Content analysis: An introduction to its methodology. Second Edition. Thousand Oaks, CA: Sage.
Maxwell, A. E. (1970). Comparing the classification of subjects by two independent judges. British Journal of Psychiatry, 116, 651-655.
Osgood, C. E. (1959). The representational model and relevant research. In I. de Sola Pool (Ed.), Trends in content analysis (pp. 33-88). Urbana: University of Illinois Press.
Potter, W. J., & Levine-Donnerstein, D. (1999). Rethinking validity and reliability in content analysis. Journal of Applied Communication Research, 27, 258-284.
Riffe, D., & Freitag, A. (1996, August). Twenty-five years of content analyses in Journalism & Mass Communication Quarterly. Paper presented to the annual convention of the Association for Education in Journalism and Mass Communication, Anaheim, CA, cited in D. Riffe, S. Lacy, & F. G. Fico (1998). Analyzing media messages. Mahwah, NJ: Erlbaum.
Scott, W. A. (1955). Reliability of content analysis: The case of nominal scale coding. Public Opinion Quarterly, 19, 321-325.
Zwick, R. (1988). Another look at interrater agreement. Psychological Bulletin, 103, 347-387.

Sorted by citations

Items 901–1,000 of 1,010

Governance mechanisms enabling inter-organizational adaptation: Lessons from grand challenge R&D programs
Hayter, C. S., & Link, A. N. (2020) · Science and Public Policy · doi:10.1093/scipol/scaa003 · Cited by 20
Are there non-verbal signals of guilt?
Julle-Danière, É., Whitehouse, J., Mielke, A., Vrij, A., Gustafsson, E., Micheletta, J., & Waller, B. M. (2020) · PLoS ONE · doi:10.1371/journal.pone.0231756 · Cited by 22
The Use of Stigmatizing Messaging in Anti-Obesity Communications Campaigns: Quantification of Obesity Stigmatization
Turner, M. M., Ford, L., Somerville, V., Javellana, D., Day, K. R., & Lapinski, M. K. (2020) · Communication Reports · doi:10.1080/08934215.2020.1793375 · Cited by 21
Smart Analysis of Economics Sentiment in Spanish Based on Linguistic Features and Transformers
García-Díaz, J. A., García‐Sánchez, F., & Valencia‐García, R. (2023) · IEEE Access · doi:10.1109/access.2023.3244065 · Cited by 21
Female Forces: Beauty, Brains, and a Badge
Rabe-Hemp, C. E. (2011) · Feminist Criminology · doi:10.1177/1557085111398471 · Cited by 20
Culture and innovation in SMEs: the intellectual structure of research for further inquiry
González‐Loureiro, M., Sousa, M. J., & Pinto, H. (2017) · European Planning Studies · doi:10.1080/09654313.2017.1290052 · Cited by 20
Rethinking health workforce planning: Capturing health system social and power interactions through actor analysis
Rees, G. H., Crampton, P., Gauld, R. D. C., & MacDonell, S. G. (2018) · Futures · doi:10.1016/j.futures.2018.03.009 · Cited by 21
Extending the Privacy Calculus: The Role of Psychological Ownership
Cichy, P., Salge, T. O., & Kohli, R. (2014) · ARBOR - Bern University of Applied Sciences Repository · doi:10.24451/arbor.16074 · Cited by 20
Detecting Topic-Oriented Speaker Stance in Conversational Speech
Lai, C., Alex, B., Moore, J. D., Tian, L., Hori, T., & Francesca, G. (2019) · Interspeech (USB) · doi:10.21437/interspeech.2019-2632 · Cited by 19
Reliability assessment for remote sensing data: Beyond Cohen's kappa
Kerr, G., Fischer, C., & Reulke, R. (2015) · IEEE International Geoscience and Remote Sensing Symposium (IGARSS) · doi:10.1109/igarss.2015.7326954 · Cited by 19
Evaluating Human versus Machine Learning Performance in a LegalTech Problem
Orosz, T., Vági, R., Csányi, G. M., Nagy, D., Üveges, I., Vadász, J. P., & Megyeri, A. (2021) · Applied Sciences · doi:10.3390/app12010297 · Cited by 21
Exploring the sense of place of traditional shopping streets through Instagram’s visual images and narratives
Kusumowidagdo, A., Ujang, N. B., Rahadiyanti, M., & Ramli, N. A. (2022) · Open House International · doi:10.1108/ohi-01-2022-0009 · Cited by 19
Culinary capital of Michelin restaurants consumers: A netnographic analysis
Lee, H.-J., & Ko, J. (2022) · Journal of Hospitality and Tourism Management · doi:10.1016/j.jhtm.2022.04.011 · Cited by 19
Applications of educational data mining and learning analytics on data from cybersecurity training
Švábenský, V., Vykopal, J., Čeleda, P., & Kraus, L. (2022) · Education and Information Technologies · doi:10.1007/s10639-022-11093-6 · Cited by 19
Impact of COVID-19 on TV advertisements: informational, transformational & narrative analyses of post-COVID-19 TV advertisements
Sharma, D., & Meena, R. (2022) · Journal of Marketing Communications · doi:10.1080/13527266.2022.2098518 · Cited by 19
Mixed-methods exploration of students’ written belonging explanations from general chemistry at a selective institution
Fink, A., Young, J. D., Vuppala, N. K., & Frey, R. F. (2022) · Chemistry Education Research and Practice · doi:10.1039/d2rp00166g · Cited by 24
Stakeholder perceptions of sustainability reporting on the websites of technical universities in Ghana
Amoako, K. O., Marfo, E. O., Akwaa‐Sekyi, E. K., Amaning, N., & Yankey, N. (2022) · Benchmarking An International Journal · doi:10.1108/bij-02-2022-0104 · Cited by 19
Requirements Classification for Traceability Link Recovery
Hey, T., Keim, J., & Corallo, S. (2024) · IEEE International Requirements Engineering Conference (RE) · doi:10.1109/re59067.2024.00024 · Cited by 19
Understanding the Information Needs of Large-Scale Digital Library Users
Zavalina, O. L., & Vassilieva, E. V. (2014) · Library Resources and Technical Services · doi:10.5860/lrts.58n2.84 · Cited by 19
Automated Indicators to Assess the Creativity of Solutions to Programming Exercises
Manske, S., & Hoppe, H. U. (2014) · IEEE International Conference on Advanced Learning Technologies (ICALT) · doi:10.1109/icalt.2014.147 · Cited by 19
Evaluating students' learning gains and experiences from using nomenclature101.com
Bodé, N. E., Caron, J., & Flynn, A. B. (2016) · Chemistry Education Research and Practice · doi:10.1039/c6rp00132g · Cited by 19
An Exploration of Responsivity among Violent Offenders: Predicting Access to Treatment, Treatment Engagement and Programme Completion
O’Brien, K. M., & Daffern, M. (2016) · Psychiatry Psychology and Law · doi:10.1080/13218719.2016.1230923 · Cited by 19
Health Promotion in Practice—District Nurses׳ Experiences of Working with Health Promotion and Lifestyle Interventions Among Patients at Risk of Developing Cardiovascular Disease
Lundberg, K., Jong, M., Kristiansen, L. P., & Jong, M. (2016) · EXPLORE · doi:10.1016/j.explore.2016.12.001 · Cited by 19
Unsupervised Discovery of Character Dictionaries in Animation Movies
Somandepalli, K., Kumar, N., Guha, T., & Narayanan, S. S. (2017) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2017.2745712 · Cited by 19
Privacy, Disability, and Family: Exploring the Privacy Management Behaviors of Parents with a Child with Autism
Hays, A., & Butauski, M. (2017) · Western Journal of Communication · doi:10.1080/10570314.2017.1398834 · Cited by 19
Entheseal changes among late Holocene hunter-gatherers from the southern extreme of La Plata basin (Argentina)
Mazza, B. P. (2018) · Archaeological and Anthropological Sciences · doi:10.1007/s12520-018-0638-2 · Cited by 19
Representations of Interpersonal Interaction and Race/Ethnicity: An Examination of Prime-Time Network Television Programs
Riles, J. M., Varava, K. A., Pilny, A. N., & Tewksbury, D. H. (2018) · Journal of Broadcasting & Electronic Media · doi:10.1080/08838151.2018.1451862 · Cited by 19
The Mitcham Score: quantifying students’ descriptions of technology
Svenningsson, J. (2019) · International Journal of Technology and Design Education · doi:10.1007/s10798-019-09530-8 · Cited by 16
Peer Leader Reflections on Promoting Discussion in Peer Group-Learning Sessions: Reflective and Practiced Advice through Collaborative Annual Peer-Advice Books
Szteinberg, G., Repice, M. D., Hendrick, C., Meyerink, S., & Frey, R. F. (2020) · CBE—Life Sciences Education · doi:10.1187/cbe.19-05-0091 · Cited by 19
Hassles of cabin crew: An exploratory study
Tsaur, S., Hsu, F.-S., & Kung, L.-H. (2020) · Journal of Air Transport Management · doi:10.1016/j.jairtraman.2020.101812 · Cited by 19
Cross-national CCTV footage shows low victimization risk for bystander interveners in public conflicts.
Liebst, L. S., Philpot, R., Levine, M., & Lindegaard, M. R. (2020) · Psychology of Violence · doi:10.1037/vio0000299 · Cited by 21
Air superiority and battlefield victory
Saunders, R., & Souva, M. (2020) · Research & Politics · doi:10.1177/2053168020972816 · Cited by 19
Statistical Power in Content Analysis Designs: How Effect Size, Sample Size and Coding Accuracy Jointly Affect Hypothesis Testing – A Monte Carlo Simulation Approach.
Geiß, S. (2021) · Computational Communication Research · doi:10.5117/ccr2021.1.003.geis · Cited by 19
Disruption and Legitimacy: Big Data in Society
Ferreira, C., Merendino, A., & Meadows, M. (2021) · Information Systems Frontiers · doi:10.1007/s10796-021-10155-3 · Cited by 19
Trash or Treasure? Considerations for Future Ecological Research to Inform Oil and Gas Decommissioning
Schläppy, M., Robinson, L. M., Camilieri‐Asch, V., & Miller, K. J. (2021) · Frontiers in Marine Science · doi:10.3389/fmars.2021.642539 · Cited by 19
Children Use Non-referential Gestures in Narrative Speech to Mark Discourse Elements Which Update Common Ground
Rohrer, P. L., Florit‐Pons, J., Vilà‐Giménez, I., & Prieto, P. (2022) · Frontiers in Psychology · doi:10.3389/fpsyg.2021.661339 · Cited by 25
Barriers to Formal IT Governance Practice -- Insights from a Qualitative Study
Othman, M. F. I., & Chan, T. (2013) · Hawaii International Conference on System Sciences (HICSS) · doi:10.1109/hicss.2013.112 · Cited by 19
Better than Their Reputation? On the Reliability of Relevance Assessments with Students
Schaer, P. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-33247-0_14 · Cited by 19
Narrative Performance of Gifted African American School-Aged Children From Low-Income Backgrounds
Mills, M. T. (2014) · American Journal of Speech-Language Pathology · doi:10.1044/2014_ajslp-13-0150 · Cited by 19
“Drinking Won't Get You Thinking”: A Content Analysis of Adolescent-Created Print Alcohol Counter-advertisements
Banerjee, S. C., Greene, K. L., Hecht, M. L., Magsamen‐Conrad, K., & Elek, E. (2013) · Health Communication · doi:10.1080/10410236.2012.762826 · Cited by 19
Multidisciplinary graduate training in social research methodology and computer-assisted qualitative data analysis: a hands-on/hands-off course design
Bourque, C. J., & Bourdon, S. (2016) · Journal of Further and Higher Education · doi:10.1080/0309877x.2015.1135882 · Cited by 27
Campaign and corporate goals in conflict: Exploring company-issue congruence through a content analysis of Coca-Cola’s twitter feed
Gaither, B. M., & Austin, L. (2016) · Public Relations Review · doi:10.1016/j.pubrev.2016.07.001 · Cited by 19
The UK Stewardship Code and investee earnings quality
Lu, C., Christensen, J., Hollindale, J., & Routledge, J. (2018) · Accounting Research Journal · doi:10.1108/arj-09-2016-0116 · Cited by 19
Genre Matters: An Examination of Women Working Behind the Scenes and On-screen Portrayals in Reality and Scripted Prime-Time Programming
Lauzen, M. M., Dozier, D. M., & Cleveland, E. (2006) · Sex Roles · doi:10.1007/s11199-006-9100-5 · Cited by 19
The cultural identity of first-generation adult immigrants: A meta-analysis
Maehler, D. B., & Daikeler, J. (2024) · Self and Identity · doi:10.1080/15298868.2024.2399559 · Cited by 23
Social media influencers as catalysts for transforming risk perception, crafting destination image, and elevating culinary tourism in Iran
Babalou, Y., Maleki, A., & Gibson, H. (2025) · Journal of Vacation Marketing · doi:10.1177/13567667241306428 · Cited by 18
A comparative performance analysis of the International Classification of Functioning, Disability and Health and the Item-Perspective Classification framework for classifying the content of patient reported outcome measures
Rosa, D., MacDermid, J. C., & Klubowicz, D. (2021) · Health and Quality of Life Outcomes · doi:10.1186/s12955-021-01774-0 · Cited by 18
Involving Caregivers of Autistic Toddlers in Early Intervention: Common Practice or Exception to the Norm?
Lee, J., Kaat, A. J., & Roberts, M. Y. (2022) · American Journal of Speech-Language Pathology · doi:10.1044/2022_ajslp-21-00246 · Cited by 18
Addressing religious hate online: from taxonomy creation to automated detection
Ramponi, A., Testa, B., Tonelli, S., & Ježek, E. (2022) · PeerJ Computer Science · doi:10.7717/peerj-cs.1128 · Cited by 21
NEREL: a Russian information extraction dataset with rich annotation for nested entities, relations, and wikidata entity links
Loukachevitch, N., Artemova, E., Batura, T. V., Braslavski, P., Ivanov, V. V., Manandhar, S., Pugachev, A., Рожков, І. М., Shelmanov, A., Tutubalina, E. V., et al. (2023) · Language Resources and Evaluation · doi:10.1007/s10579-023-09674-z · Cited by 19
Metaphorical evolution: A longitudinal study of secondary school teachers’ concepts of 3D modelling and printing in education
Anđić, B., Maričić, M., Weinhandl, R., Mumcu, F., Schmidthaler, E., & Lavicza, Z. (2024) · Education and Information Technologies · doi:10.1007/s10639-023-12408-x · Cited by 18
Virtual reality roleplays for patients with depression: A user experience evaluation
Holsteg, S., Askeridis, J. M., Krajewski, J., Mildner, P., Freitag, S., Müller, T., Schnieder, S., Gieselmann, A., & Karger, A. (2024) · Internet Interventions · doi:10.1016/j.invent.2024.100713 · Cited by 19
Resilient business strategies in the German hospitality industry and its viability
Ruppenthal, T., & Rückert‐John, J. (2024) · Tourism & Management Studies · doi:10.18089/tms.20240301 · Cited by 18
Why Do We Think Politicians Are So Evasive? Insight From Theories of Equivocation and Deception, With a Content Analysis of U.S. Presidential Debates, 1996-2012
Clementson, D. E. (2015) · Journal of Language and Social Psychology · doi:10.1177/0261927x15600732 · Cited by 22
Diffuse microvascular C5b-9 deposition is a common feature in muscle and nerve biopsies from diabetic patients
Yell, P. C., Burns, D. K., Dittmar, E., White, C. L., & Cai, C. (2018) · Acta Neuropathologica Communications · doi:10.1186/s40478-018-0512-6 · Cited by 18
Digging for (Ratings) Gold: The Connection Between Investigative Journalism and Audiences
Abdenour, J., & Riffe, D. (2019) · Journalism Studies · doi:10.1080/1461670x.2019.1598887 · Cited by 18
Human–Water Infrastructure Interactions: Substituting Services Received for Bottled and Filtered Water in US Shrinking Cities
Yang, E., & Faust, K. M. (2019) · Journal of Water Resources Planning and Management · doi:10.1061/(asce)wr.1943-5452.0001126 · Cited by 19
Compliance Cautions: Investigating Security Issues Associated with U.S. Digital-Security Standards
Stevens, R., Dykstra, J., Everette, W. K., Chapman, J., Bladow, G., Farmer, A., Halliday, K., & Mazurek, M. L. (2020) · Network and Distributed System Security Symposium · doi:10.14722/ndss.2020.24003 · Cited by 18
The cultural identity of first-generation immigrant children and youth: Insights from a meta-analysis
Maehler, D. B., Daikeler, J., Ramos, H., Husson, C., & Nguyen, T. A. (2020) · Self and Identity · doi:10.1080/15298868.2020.1765857 · Cited by 18
Discussing surgical innovation with patients: a qualitative study of surgeons’ and governance representatives’ views
Zahra, J., Paramasivan, S., Blencowe, N., Cousins, S., AVERY, K. N. L., Mathews, J. A., Main, B., McNair, A., Hinchliffe, R. J., Blazeby, J. M., et al. (2020) · BMJ Open · doi:10.1136/bmjopen-2019-035251 · Cited by 18
Is Digital News Really that Digital? An Analysis of How Online News Sites in the UK use Digital Affordances to Enhance Their Reporting
Harmer, E., & Southern, R. (2020) · Journalism Studies · doi:10.1080/1461670x.2020.1831397 · Cited by 18
Enzymatic Debridement for Burn Wound Care: Interrater Reliability and Impact of Experience in Post-intervention Therapy Decision
Siegwart, L. C., Böcker, A., Diehm, Y. F., Kotsougiani, D., Erdmann, S., Ziegler, B., Kneser, U., Hirche, C. R., & Fischer, S. (2020) · Journal of Burn Care & Research · doi:10.1093/jbcr/iraa218 · Cited by 19
Representation Problems in Linguistic Annotations : Ambiguity, Variation, Uncertainty, Error and Bias
Beck, C., Booth, H., El‐Assady, M., & Butt, M. (2020) · Ghent University Academic Bibliography (Ghent University) · Cited by 18
Distrust as a Hazard for Future Sustainable Mobility Planning. Rethinking Employees’ Vulnerability When Introducing New Information and Communication Technologies in Local Authorities
Nienaber, A., Spundflasch, S., Soares, A. E., & Woodcock, A. (2021) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2020.1860547 · Cited by 18
Assessment of COVID-19 Website Communication in Languages Other than English by Local Health Departments in the United States
Kusters, I. S., Dean, J. M., Gutierrez, A. M., Sommer, M., & Klyueva, A. (2021) · Health Communication · doi:10.1080/10410236.2021.2017109 · Cited by 18
Swiss Job Market Monitor: A Rich Source of Demand-Side Micro Data of the Labour Market
Buchmann, M. C., Buchs, H., Busch, F., Clematide, S., Gnehm, A.-S., & Müller, J. (2022) · European Sociological Review · doi:10.1093/esr/jcac002 · Cited by 18
Thanks for joining our life: Intimacy as performativity on YouTube parenting vlogs
Vizcaíno-Verdú, A., Moreno, P. D. C., & Jaramillo-Dent, D. (2022) · El Profesional de la Informacion · doi:10.3145/epi.2022.jul.07 · Cited by 18
Institutional work battles in the sharing economy: Unveiling actors and discursive strategies in media discourse
Lehmann, J., Weber, F., Waldkirch, M., Graf‐Vlachy, L., & Konig, A. S. (2022) · Technological Forecasting and Social Change · doi:10.1016/j.techfore.2022.122002 · Cited by 20
Global Journalism in Decision-Making Moments: A Case Study of Canadian and American Television Coverage of the 2009 United Nations Framework Convention on Climate Change in Copenhagen
Konieczna, M., Mattis, K., Tsai, J.-Y., Liang, X., & Dunwoody, S. (2014) · Environmental Communication · doi:10.1080/17524032.2014.909509 · Cited by 18
Eliciting positive user experiences with self-service kiosks: pursuing possibilities
Günay, A., & Erbuğ, Ç. (2014) · Behaviour and Information Technology · doi:10.1080/0144929x.2014.937459 · Cited by 18
On the Problem of Theoretical Terms in Empirical Computational Linguistics
Riezler, S. (2013) · Computational Linguistics · doi:10.1162/coli_a_00182 · Cited by 18
Persuading to Pay: Exploring the What and Why in Crowdfunded Journalism
Ladson, N., & Lee, A. M. (2017) · The International Journal on Media Management · doi:10.1080/14241277.2017.1298110 · Cited by 18
A longitudinal analysis on the perspectives of major world newspapers on the Three Gorges Dam project during 1982–2015
Wu, S., Huang, S., Wei, Y., Duffield, C. F., Tang, W., Zhao, Y., & Zheng, H. (2017) · Water Science & Technology Water Supply · doi:10.2166/ws.2017.088 · Cited by 18
Grading Prenatal Hydronephrosis from Ultrasound Imaging Using Deep Convolutional Neural Networks
Dhindsa, K., Smail, L. C., McGrath, M., Braga, L. H., Becker, S., & Sonnadara, R. (2018) · Canadian Conference on Computer and Robot Vision · doi:10.1109/crv.2018.00021 · Cited by 18
Learners’ Representation of their Affective Domain through Figurative Language in a Web‐Based Learning Environment
Manca, S., & Delfino, M. M. (2007) · Distance Education · doi:10.1080/01587910701305293 · Cited by 18
Public health framing in local media coverage of crashes involving pedestrians or bicyclists in Hawai‘i, 2019: A content analysis
Keliikoa, L. B., Thompson, M. D., Johnson, C. J., Cacal, S. L., Pirkle, C. M., & Sentell, T. L. (2022) · Transportation Research Interdisciplinary Perspectives · doi:10.1016/j.trip.2021.100525 · Cited by 17
Preschoolers’ choices of television characters as sources of information: Effects of character type, format, and topic domain
Snyder, M. N., & Mares, M.-L. (2021) · Journal of Experimental Child Psychology · doi:10.1016/j.jecp.2020.105034 · Cited by 17
The Changing German Voter
Schmitt‐Beck, R., Roßteutscher, S., Schoen, H., Weßels, B., & Wolf, C. (2022) · Oxford University Press eBooks · doi:10.1093/oso/9780198847519.003.0015 · Cited by 17
Robotic roles in education: A systematic review based on a proposed framework of the learner-robot relationships
Xu, W., & Ouyang, F. (2025) · Educational Research Review · doi:10.1016/j.edurev.2025.100685 · Cited by 21
Federated Entity Search Using On-the-Fly Consolidation
Herzig, D. M., Mika, P., Blanco, R., & Tran, T. P. (2013) · Lecture notes in computer science · doi:10.1007/978-3-642-41335-3_11 · Cited by 17
Did online publishers “get it right”? Using a naturalistic search strategy to review cognitive health promotion content on internet webpages
Hunter, P. V., Delbaere, M., O’Connell, M. E., Cammer, A. L., Seaton, J., Friedrich, T. E., & Fick, F. (2017) · BMC Geriatrics · doi:10.1186/s12877-017-0515-3 · Cited by 17
‘Man-eaters’ in the Media: Representation of Human-leopard Interactions in India Across Local, National, and International Media
Crown, C. A., & Doubleday, K. F. (2017) · Conservation and Society · doi:10.4103/cs.cs_15_92 · Cited by 17
Positive and Negative Social Media Experiences Among Young Adults With and Without Depressive Symptoms
Shensa, A. S., Sidani, J. E., Hoffman, B. L., Escobar-Viera, C. G., Melcher, E. M., Primack, B. A., Myers, S. P., & Burke, J. G. (2020) · Journal of Technology in Behavioral Science · doi:10.1007/s41347-020-00175-2 · Cited by 17
Home Nation First, but to What Degree?: Nationalism in Chinese Central Television’s Broadcasts of the 2018 Winter Olympics
Guo, S., & Xu, Q. (2021) · Communication & Sport · doi:10.1177/21674795211044188 · Cited by 17
The effects of a game-enhanced learning intervention on foreign language learning
Aydın, S. M., & Çakır, N. A. (2022) · Educational Technology Research and Development · doi:10.1007/s11423-022-10141-9 · Cited by 17
What Are Patients Told About Innovative Surgical Procedures? A Qualitative Synthesis of 7 Case Studies in the United Kingdom
Elliott, D., Ochieng, C., Zahra, J., McNair, A., Main, B., Skilton, A., Blencowe, N., Cousins, S., Paramasivan, S., Hoffmann, C., et al. (2022) · Annals of Surgery · doi:10.1097/sla.0000000000005714 · Cited by 17
Is everything quantum ‘spooky and weird’? An exploration of popular communication about quantum science and technology in TEDx talks
Meinsma, A. L., Kristensen, S. W., Reijnierse, W. G., Smeets, I., & Cramer, J. (2023) · Quantum Science and Technology · doi:10.1088/2058-9565/acc968 · Cited by 27
The role of substance use treatment in reducing stigma after release from incarceration: A qualitative analysis
Moore, K. E., Wyatt, J. P., Phillips, S. M., Burke, C., Bellamy, C. D., & McKee, S. A. (2023) · Health & Justice · doi:10.1186/s40352-023-00225-w · Cited by 21
Fast fashion consumer engagement on Instagram: a case study
Bonilla-Quijada, M., Arriaga, J. L. D. O., Domingo, D. A., & Alcón, J. R. I. (2024) · Cogent Business & Management · doi:10.1080/23311975.2024.2322111 · Cited by 17
The use of Islamic hedging instruments as non-speculative risk management tools
Mohamad, S., Othman, J. B., Roslin, R. M., & Lehner, O. M. (2014) · Venture Capital · doi:10.1080/13691066.2014.922824 · Cited by 17
Police legitimacy under the spotlight: media coverage of police performance in the face of a high terrorism threat
Sela‐Shayovitz, R. (2014) · Journal of Experimental Criminology · doi:10.1007/s11292-014-9213-8 · Cited by 17
Operationalizing Heedful Interrelating: How Attending, Responding, and Feeling Comprise Coordinating and Predict Performance in Self-Managing Teams
Stephens, J. P., & Lyddy, C. J. (2016) · Frontiers in Psychology · doi:10.3389/fpsyg.2016.00362 · Cited by 18
A citizen science approach to monitoring bleaching in the zoantharian Palythoa tuberculosa
Parkinson, J. E., Yang, S., Kawamura, I., Byron, G., Todd, P. A., & Reimer, J. D. (2016) · PeerJ · doi:10.7717/peerj.1815 · Cited by 17
Talking it through: communication sequences in negotiation
Adair, W. L., & Loewenstein, J. (2013) · Edward Elgar Publishing eBooks · doi:10.4337/9781781005903.00022 · Cited by 17
Improving relevance in a content pipeline via syntactic generalization
Galitsky, B. A. (2016) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2016.11.001 · Cited by 17
Patterns of schematic structure and strategic features in newspaper editorials: A comparative study of American and Malaysian editorials
Zarza, S., & Tan, H. (2016) · Discourse & Communication · doi:10.1177/1750481316674754 · Cited by 17
The Representation of Altruistic and Egoistic Motivations in Popular Music over 60 Years
Hahn, L., Tamborini, R., Klebig, B., Novotny, E. R., Grall, C., Höfer, M., & Lee, H. (2018) · Communication Studies · doi:10.1080/10510974.2018.1447493 · Cited by 17
Reconceptualizing TQM in service ecosystems: an integrated framework
Polese, F., Vesci, M., Troisi, O., & Grimaldi, M. (2018) · International Journal of Quality and Service Sciences · doi:10.1108/ijqss-10-2017-0087 · Cited by 18
Portrayals of Vaccination in Entertainment Television: A Content Analysis
McClaran, N., & Rhodes, N. D. (2020) · Health Communication · doi:10.1080/10410236.2020.1749356 · Cited by 17
Detecting behavioural change in conversation: procedures and preliminary data
Azios, J. H., Archer, B., & Lee, J. B. (2020) · Aphasiology · doi:10.1080/02687038.2020.1812031 · Cited by 17

Authors: Krippendorff, K.
Corresponding author: Krippendorff, K.
ORCID: Krippendorff, K.
Venue: Human Communication Research
Publisher: Oxford University Press (OUP)
Published: 2004-07-01
Volume: 30
Issue: 3
Pages: 411-433
ISSN: 0360-3989 (ISSN-L)
1468-2958
DOI: 10.1093/hcr/30.3.411
OpenAlex: W4243989635
Cited by: 1,027
References: 0
Citations by year: 2026: 42 · 2025: 100 · 2024: 108 · 2023: 108 · 2022: 124 · 2021: 135 · 2020: 105 · 2019: 57 · 2018: 44 · 2017: 29 · 2016: 41 · 2015: 25 · 2014: 26 · 2013: 21 · 2012: 15
FWCI: 5.93
Type: journal article
Language: en
Open access: green
repository.upenn.edu
Licence: doi.wiley.com/10.1002/tdm_license_1 (tdm, 4079 d)
Locations: Human Communication Research (publishedVersion)
ScholarlyCommons (University of Pennsylvania) (submittedVersion)
Topics: Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Reliability and Agreement in Measurement
Social Sciences › General Social Sciences › Computational and Text Analysis Methods
Social Sciences › Decision Sciences › Management Science and Operations Research › Psychometric Methodologies and Testing
Keywords: inter-rater reliability, reliability coefficients, interobserver agreement, reporting reliability

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Reliability in content analysis: some common misconceptions and recommendations.

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Self-check

Citations

Cites

Cited by

Related

Metadata

Record

About these checks