Cited by 1,027Reliability and Agreement in MeasurementComputational and Text Analysis MethodsPsychometric Methodologies and Testing

The DOI matches its Crossref record; 5 of the 5 references it cites were found in a registry or an index.

Reliability in content analysis: some common misconceptions and recommendations.

Open access

Krippendorff, K.·2004·Human Communication Research

DOI 10.1093/hcr/30.3.411

Open this paper in CoChat — free account Open PDF

Abstract

In a recent article published in this journal, Lombard, Snyder-Duch, and Bracken (2002) surveyed 200 content analyses for their reporting of reliability tests; compared the virtues and drawbacks of five popular reliability measures; and proposed guidelines and standards for their use. Their discussion revealed that numerous misconceptions circulate in the content analysis literature regarding how these measures behave and can aid or deceive content analysts in their effort to ensure the reliability of their data. This paper proposes three conditions for statistical measures to serve as indices of the reliability of data and examines the mathematical structure and the behavior of the five coefficients discussed by the authors, plus two others. It compares common beliefs about these coefficients with what they actually do and concludes with alternative recommendations for testing reliability in content analysis and similar data-making efforts.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Integrity signals

Registry

Crossref · 10.1093/hcr/30.3.411

Cites

0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

The publisher hasn’t deposited this paper’s reference list.

Research works printed but not linked: 13

Bennett, E. M., Alpert, R., & Goldstein, A. C. (1954). Communications through limited response questioning. Public Opinion Quarterly, 18, 303-308.
Craig, R. T. (1981). Generalization of Scott"s index of intercoder agreement. Public Opinion Quarterly, 45, 260-264.
Goodman, L. A, & Kruskal, W. H. (1954). Measures of association for cross classifications. Journal of the American Statistical Association, 49, 732-764.
Holsti, O. R. (1969). Content analysis for the social sciences and humanities. Reading, MA: Addison- Wesley.
Hughes, M. A., & Garrett, D. E. (1990). Intercoder reliability estimation -Approaches in marketing: A generalizability theory framework for quantitative data. Journal of Marketing Research, 27, 185- 195.
Krippendorff, K. (1970). Bivariate agreement coefficients for reliability data. In E. R. Borgatta & G. W. Bohrnstedt (Eds.), Sociological methodology 1970 (pp. 139-150). San Francisco, CA: Jossey Bass.
Krippendorff, K. (2004). Content analysis: An introduction to its methodology. Second Edition. Thousand Oaks, CA: Sage.
Maxwell, A. E. (1970). Comparing the classification of subjects by two independent judges. British Journal of Psychiatry, 116, 651-655.
Osgood, C. E. (1959). The representational model and relevant research. In I. de Sola Pool (Ed.), Trends in content analysis (pp. 33-88). Urbana: University of Illinois Press.
Potter, W. J., & Levine-Donnerstein, D. (1999). Rethinking validity and reliability in content analysis. Journal of Applied Communication Research, 27, 258-284.
Riffe, D., & Freitag, A. (1996, August). Twenty-five years of content analyses in Journalism & Mass Communication Quarterly. Paper presented to the annual convention of the Association for Education in Journalism and Mass Communication, Anaheim, CA, cited in D. Riffe, S. Lacy, & F. G. Fico (1998). Analyzing media messages. Mahwah, NJ: Erlbaum.
Scott, W. A. (1955). Reliability of content analysis: The case of nominal scale coding. Public Opinion Quarterly, 19, 321-325.
Zwick, R. (1988). Another look at interrater agreement. Psychological Bulletin, 103, 347-387.

Sorted by citations

Items 501–600 of 1,010

Road Safety Differences between Priority-Controlled Intersections and Right-Hand Priority Intersections
Ceunynck, T. D., Polders, E., Daniëls, S., Hermans, E., Brijs, T., & Wets, G. (2013) · Transportation Research Record Journal of the Transportation Research Board · doi:10.3141/2365-06 · Cited by 40
How does the stigma of problem gambling influence help-seeking, treatment and recovery? a view from the counselling sector
Hing, N., Nuske, E., Gainsbury, S., Russell, A. M. T., & Breen, H. M. (2016) · International Gambling Studies · doi:10.1080/14459795.2016.1171888 · Cited by 40
Inter‐Rater Reliability of Total Body Score—A Scale for Quantification of Corpse Decomposition
Nawrocka, M., Frątczak‐Łagiewska, K., & Matuszewski, S. (2016) · Journal of Forensic Sciences · doi:10.1111/1556-4029.13105 · Cited by 41
Adapting supply chain management strategies to security – an analysis of existing gaps and recommendations for improvement
Urciuoli, L., & Hintsa, J. (2016) · International Journal of Logistics Research and Applications · doi:10.1080/13675567.2016.1219703 · Cited by 40
Dissemination of Social Accounting Information: A Bibliometric Review
Rodrigues, M. M. M., Alves, M. D. C. G., Oliveira, C., Vale, V. T., Vale, J., & Silva, R. (2021) · Economies · doi:10.3390/economies9010041 · Cited by 40
The Biggest Challenges Resulting from the COVID-19 Pandemic on Gender-Related Work from Home in Biomedical Fields—World-Wide Qualitative Survey Analysis
Bezak, E., Carson‐Chahhoud, K., Marcu, L. G., Stoeva, M. S., Lhotská, L., Barabino, G. A., Ibrahim, F., Kaldoudi, E., Lim, S., da Silva, A. M. M., et al. (2022) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph19053109 · Cited by 40
Recognition and Respect Revisited: Portrayals of Age and Gender in Prime-Time Television
Lauzen, M. M., & Dozier, D. M. (2005) · Mass Communication & Society · doi:10.1207/s15327825mcs0803_4 · Cited by 40
News of disparity: Content analysis of news coverage of African American healthcare inequalities in the USA, 1994–2004
Taylor-Clark, K. A., Mebane, F. E., SteelFisher, G. K., & Blendon, R. J. (2007) · Social Science & Medicine · doi:10.1016/j.socscimed.2007.03.039 · Cited by 40
Alcohol harm reduction advertisements: a content analysis of topic, objective, emotional tone, execution and target audience
Dunstone, K., Brennan, E., Slater, M. D., Dixon, H., Durkin, S. J., Pettigrew, S., & Wakefield, M. A. (2017) · BMC Public Health · doi:10.1186/s12889-017-4218-7 · Cited by 40
Students’ uncertainty management in the college classroom
Sollitto, M., Brott, J. B., Cole, C., Gil, E., & Selim, H. (2017) · Communication Education · doi:10.1080/03634523.2017.1372586 · Cited by 41
‘Who Likes What and, Why?’ Insights into Modeling Users’ Personality Based on Image ‘Likes’
Guntuku, S. C., Zhou, J. T., Roy, S., Lin, W., & Tsang, I. W.-H. (2016) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2016.2581168 · Cited by 42
AA attendance and abstinence for dually diagnosed patients: a meta‐analytic review
Tonigan, J. S., Pearson, M. R., Magill, M., & Hagler, K. J. (2018) · Addiction · doi:10.1111/add.14268 · Cited by 39
Drivers’ attitudes, preference, and acceptance of in-vehicle anger intervention systems and their relationships to demographic and personality characteristics
Li, S., Zhang, T., Liu, N., Zhang, W., Tao, D., & Wang, Z. (2019) · International Journal of Industrial Ergonomics · doi:10.1016/j.ergon.2019.102899 · Cited by 39
The Impact of the Sustainability Principles on the Strategic Planning and Reporting of Universities. An Exploratory Study on a Qualified Italian Sample
Nauta, P. D., Iannuzzi, E., Milone, M., & Nigro, C. (2020) · Sustainability · doi:10.3390/su12187269 · Cited by 39
Pinning for Profit? Examining Elementary Preservice Teachers' Critical Analysis of Online Social Studies Resources about Black History.
Rodríguez, N. N., Brown, M. G., & Vickery, A. E. (2020) · Contemporary issues in technology and teacher education · Cited by 39
Explaining the Why in #WhyIDidntReport: An Examination of Common Barriers to Formal Disclosure of Sexual Assault in College Students
Griffin, V. W., Wentz, E. A., & Meinert, E. K. (2021) · Journal of Interpersonal Violence · doi:10.1177/08862605211016343 · Cited by 43
Sustainability and profitability efficiencies: the moderating role of corporate social responsibility
Kuo, K.-C., Lu, W., & Ganbaatar, O. (2022) · International Transactions in Operational Research · doi:10.1111/itor.13152 · Cited by 39
Examining American and Chinese Internet Users' Contextual Privacy Preferences of Behavioral Advertising
Wang, Y., Xia, H., & Huang, Y. K. (2016) · ACM Conference on Computer Supported Cooperative Work (CSCW) · doi:10.1145/2818048.2819941 · Cited by 42
LENS: A Learnable Evaluation Metric for Text Simplification
Maddela, M., Dou, Y., Heineman, D., & Xu, W. H. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.905 · Cited by 52
Machine Learning Techniques in Optical Networks: A Systematic Mapping Study
Villa, G., Tipantuña, C. J., Guamán, D. S., Arévalo, G. V., & Arguero, B. (2023) · IEEE Access · doi:10.1109/access.2023.3312387 · Cited by 41
Evaluating Generative AI Psychotherapy Chatbots Used by Youth: Cross-Sectional Study
Sobowale, K., Humphrey, D. K., & Zhao, S. Y. (2025) · JMIR Mental Health · doi:10.2196/79838 · Cited by 42
When ethnic humor goes digital
Boxman-Shabtai, L., & Shifman, L. (2013) · New Media & Society · doi:10.1177/1461444813506972 · Cited by 38
Sex, Love, and Risk-n-Responsibility: A Content Analysis of Entertainment Television
Carpentier, F. R. D., Stevens, E. M., Wu, L. H., & Seely, N. (2017) · Mass Communication & Society · doi:10.1080/15205436.2017.1298807 · Cited by 38
Job insecurity and creativity: The buffering effect of self‐affirmation and work‐affirmation
Jiang, L. (2018) · Journal of Applied Social Psychology · doi:10.1111/jasp.12519 · Cited by 41
Subjective versus objective knowledge of online safety/dangers as predictors of children’s perceived online safety and attitudes towards e-safety education in the United Kingdom
Macaulay, P. J. R., Boulton, M. J., Betts, L. R., Boulton, L., Camerone, E. M., Down, J., Hughes, J., Kirkbride, C., & Kirkham, R. (2019) · Journal of Children and Media · doi:10.1080/17482798.2019.1697716 · Cited by 43
Qualitative Investigation to Identify the Knowledge and Skills That U.S.-Trained Doctoral Chemists Require in Typical Chemistry Positions
Cui, Q., & Harshman, J. T. (2020) · Journal of Chemical Education · doi:10.1021/acs.jchemed.9b01027 · Cited by 38
Enhancing Event Log Quality: Detecting and Quantifying Timestamp Imperfections
Fischer, D., Goel, K., Andrews, R., van Dun, C., Wynn, M. T., & Röglinger, M. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58666-9_18 · Cited by 38
This Is Not What We Ordered: Exploring Why Biased Search Result Rankings Affect User Attitudes on Debated Topics
Draws, T., Tintarev, N., Gadiraju, U., Bozzon, A., & Timmermans, B. (2021) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3404835.3462851 · Cited by 51
Remote Patient Monitoring Program for COVID-19 Patients Following Hospital Discharge: A Cross-Sectional Study
Bouabida, K., Malas, K., Talbot, A., Desrosiers, M.-È., Lavoie, F., Lebouché, B., Taguemout, M., Rafie, E., Lessard, D., & Pomey, M. (2021) · Frontiers in Digital Health · doi:10.3389/fdgth.2021.721044 · Cited by 38
Identifying skills, qualifications, and attributes expected to do a PhD
Mantai, L., & Marrone, M. (2022) · Studies in Higher Education · doi:10.1080/03075079.2022.2061444 · Cited by 38
Senior charge nurses’ leadership behaviours in relation to hospital ward safety: A mixed method study
Agnew, Ç., & Flin, R. H. (2013) · International Journal of Nursing Studies · doi:10.1016/j.ijnurstu.2013.10.001 · Cited by 38
Argumentation Theory in Education Studies: Coding and Improving Students’ Argumentative Strategies
Macagno, F., Mayweg-Paus, E., & Kuhn, D. (2014) · Topoi · doi:10.1007/s11245-014-9271-6 · Cited by 43
Using photovoice and emotional maps to understand transitional urban neighborhoods
Meenar, M., & Mandarano, L. A. (2021) · Cities · doi:10.1016/j.cities.2021.103353 · Cited by 37
Inter-observer Variability of Clinical Activity Score: Assessments in Patients With Thyroid Eye Disease
Perros, P., Žarković, M., Pearce, S. H. S., Razvi, S., Kolli, H., & Dickinson, A. J. (2023) · American Journal of Ophthalmology · doi:10.1016/j.ajo.2023.03.027 · Cited by 39
Investigating Mobile-Assisted Language Learning Apps: Babbel, Memrise, and Duolingo as a Case Study
Essafi, M., Belfakir, L., & Moubtassime, M. (2024) · Journal of Curriculum and Teaching · doi:10.5430/jct.v13n2p197 · Cited by 37
Assessing the Reporting of Reliability in Published Content Analyses: 1985–2010
Lovejoy, J., Watson, B. R., Lacy, S. R., & Riffe, D. (2014) · Communication Methods and Measures · doi:10.1080/19312458.2014.937528 · Cited by 43
The impact of anonymity and issue controversiality on the quality of online discussion
Berg, J. (2016) · Journal of Information Technology & Politics · doi:10.1080/19331681.2015.1131654 · Cited by 37
Quantitative indicators for social sustainability assessment of society and product responsibility aspects in supply chains
Popović, T., Krasławski, A., Barbosa‐Póvoa, A. P., & Carvalho, A. I. C. D. S. G. (2017) · JOURNAL OF INTERNATIONAL STUDIES · doi:10.14254/2071-8330.2017/10-4/1 · Cited by 37
Industry self-regulation of food advertisement to children: Compliance versus effectiveness of the EU Pledge
Landwehr, S. C., & Hartmann, M. (2020) · Food Policy · doi:10.1016/j.foodpol.2020.101833 · Cited by 37
The effects of migrant background and parent gender on child protection decision-making: An intersectional analysis
Middel, F., López, M. L., Fluke, J., & Grietens, H. W. E. (2020) · Child Abuse & Neglect · doi:10.1016/j.chiabu.2020.104479 · Cited by 43
Online decision aids: the role of decision-making styles and decision-making stages
Virdi, P., Kalro, A. D., & Sharma, D. (2020) · International Journal of Retail & Distribution Management · doi:10.1108/ijrdm-02-2019-0068 · Cited by 37
Encouraging Greater Student Inquiry Engagement in Science Through Motivational Support by Online Scientist‐Mentors
Scogin, S. C., & Stuessy, C. L. (2015) · Science Education · doi:10.1002/sce.21145 · Cited by 40
A Cross-Cultural and Cross-Media Comparison of Female Nudity in Advertising
Paek, H., & Nelson, M. R. (2007) · Journal of Promotion Management · doi:10.1300/j057v13n01_10 · Cited by 37
Combined 5‐minute double‐echo in steady‐state with separated echoes and 2‐minute proton‐density‐weighted 2D FSE sequence for comprehensive whole‐joint knee MRI assessment
Chaudhari, A. S., Stevens, K. J., Sveinsson, B., Wood, J. P., Beaulieu, C. F., Oei, E. H. G., Rosenberg, J., Kogan, F., Alley, M. T., Gold, G. E., et al. (2018) · Journal of Magnetic Resonance Imaging · doi:10.1002/jmri.26582 · Cited by 37
Gender, Race, and Risk: Intersectional Risk Management in the Sale of Sex Online
Moorman, J. D., & Harrison, K. (2015) · The Journal of Sex Research · doi:10.1080/00224499.2015.1065950 · Cited by 42
Student-to-Student Confirmation in the College Classroom: An Initial Investigation of the Dimensions and Outcomes of Students’ Confirming Messages
Johnson, Z. D., & LaBelle, S. (2015) · Communication Education · doi:10.1080/03634523.2015.1058961 · Cited by 36
Predictors and Effects of Anti-Obesity Message Fatigue: A Thought-Listing Analysis
So, J., & Alam, N. (2018) · Health Communication · doi:10.1080/10410236.2018.1434736 · Cited by 39
Between cheap talk and epistocracy: The logic of interest group access in the European Parliament's committee hearings
Coen, D., & Katsaitis, A. (2018) · Public Administration · doi:10.1111/padm.12406 · Cited by 36
Validation of Dried Blood Spots for Maternal Biomonitoring of Nonessential Elements in an Artisanal and Small-Scale Gold Mining Area of Tanzania
Nyanza, E. C., Dewey, D., Bernier, F. P., Manyama, M. F., Hatfield, J. M., & Martin, J. W. (2019) · Environmental Toxicology and Chemistry · doi:10.1002/etc.4420 · Cited by 36
Managing Uncertainty in Paramedics’ Decision Making
Harenčárová, H. (2016) · Journal of Cognitive Engineering and Decision Making · doi:10.1177/1555343416674814 · Cited by 44
Conflicting Evidence or Conflicting Opinions? Two-Sided Expert Discussions Contribute to Experts’ Trustworthiness
Mayweg-Paus, E., & Jucks, R. (2017) · Journal of Language and Social Psychology · doi:10.1177/0261927x17716102 · Cited by 36
Content, Quality, and Assessment Tools of Physician-Rating Websites in 12 Countries: Quantitative Analysis
Rothenfluh, F., & Schulz, P. J. (2018) · Journal of Medical Internet Research · doi:10.2196/jmir.9105 · Cited by 36
Delving into possible missing links for attainment of food security in Central Malawi: farmers’ perceptions and long term dynamics in maize (Zea mays L.) production
Nyirenda, H., Mwangomba, W., & Nyirenda, E. M. (2021) · Heliyon · doi:10.1016/j.heliyon.2021.e07130 · Cited by 35
“Refusing appearance anxiety”: A content analysis of body positive posts on Chinese social media
Lang, M., Chen, P., Li, X., & Ye, Y. (2023) · Body Image · doi:10.1016/j.bodyim.2023.04.002 · Cited by 35
The relation between questionnaire-measured self-reported emotional eating and disordered eating behaviors: A meta-analysis of nearly three decades of research
Barnhart, W. R., Kalantzis, M. A., Gaggiano, C. M., & Braden, A. L. (2024) · Appetite · doi:10.1016/j.appet.2024.107343 · Cited by 37
A Multi-Institutional Comparison of Dynamic Contrast-Enhanced Magnetic Resonance Imaging Parameter Calculations
Cooperative, J. H. A. N. R.-M. D., Ger, R. B., Mohamed, A., Awan, M. J., Ding, Y., Li, K., Fave, X. J., Beers, A. L., Driscoll, B., Elhalawani, H. M., et al. (2017) · Scientific Reports · doi:10.1038/s41598-017-11554-w · Cited by 35
#SELFIES at the 2016 Rio Olympics: Comparing Self-Representations of Male and Female Athletes from the U.S. And China
Xu, Q., & Armstrong, C. L. (2019) · Journal of Broadcasting & Electronic Media · doi:10.1080/08838151.2019.1621138 · Cited by 35
The Impact of COVID-19 on Financial Statements Results and Disclosure: First Insights from Italian Listed Companies
Tibiletti, V., Marchini, P. L., Gamba, V., & Todaro, D. L. (2021) · Universal Journal of Accounting and Finance · doi:10.13189/ujaf.2021.090106 · Cited by 35
NPO Web-Based Accountability: How Can We Know If NPOs Are Doing Good Things?
Chu, V., & Luke, B. (2021) · Nonprofit and Voluntary Sector Quarterly · doi:10.1177/08997640211062856 · Cited by 35
Exploration of the (Interrater) Reliability and Latent Factor Structure of the Alcohol Use Disorders Identification Test (AUDIT) and the Drug Use Disorders Identification Test (DUDIT) in a Sample of Dutch Probationers
Hildebrand, M., & Noteborn, M. G. C. (2015) · Substance Use & Misuse · doi:10.3109/10826084.2014.998238 · Cited by 35
Nüfuz Pazarlamasında (Influencer Marketing) Mesaj Stratejileri
AKTAŞ, A., & ŞENER, G. (2019) · Erciyes İletişim Dergisi · doi:10.17680/erciyesiletisim.477592 · Cited by 35
Leaders and elites: portrayals of smoking in popular films: Table 1
Dozier, D. M., Lauzen, M. M., Day, C. A., Payne, S., & Tafoya, M. R. (2005) · Tobacco Control · doi:10.1136/tc.2003.006205 · Cited by 35
Proceedings of the First Workshop on NLP and Computational Social Science
Bamnan, D., Doğruöz, A. S., Eisenstein, J., Hovy, D., Jurgens, D., O'Connor, B., Oh, A., Tsur, O., & Volkova, S. (2016) · doi:10.18653/v1/w16-56 · Cited by 36
Authoritarian Environmentalism as Reflected in the Journalistic Sourcing of Climate Change Reporting in China
Guo, J., Huang, X., & Fang, K. (2023) · Environmental Communication · doi:10.1080/17524032.2023.2223774 · Cited by 37
Explaining the interplay of value creation and value appropriation in strategic alliances: A developmental perspective
Ferrigno, G., Martin, X., & Dagnino, G. B. (2023) · International Journal of Management Reviews · doi:10.1111/ijmr.12351 · Cited by 27
Nexus of circular economy R0 to R9 principles in integrated reporting: Insights from a multiple case study comparison
Nazir, S., & Doni, F. (2024) · Business Strategy and the Environment · doi:10.1002/bse.3684 · Cited by 34
New York City Bed Bug Crisis as Framed by Tourists on Tripadvisor
Liu-Lastres, B., Pennington-Gray, L. A., Donohoe, H., & Omodior, O. (2015) · Tourism Analysis · doi:10.3727/108354215x14265319207597 · Cited by 34
Making Everyday Microaggressions : An Exploratory Experimental Vignette Study on the Presence and Power of Racial Microaggressions
Hughey, M. W., Rees, J. H., Goss, D. R., Rosino, M. L., & Lesser, E. (2017) · Sociological Inquiry · doi:10.1111/soin.12167 · Cited by 44
“Miscommunication” and Undergraduate Women’s Conceptualizations of Sexual Assault: A Qualitative Analysis
Dardis, C. M., Kraft, K. M., & Gidycz, C. A. (2017) · Journal of Interpersonal Violence · doi:10.1177/0886260517726412 · Cited by 34
From China to the West: why manufacturing locates in developed countries
Ancarani, A., Mauro, C. D., Virtanen, Y., & You, W. (2020) · International Journal of Production Research · doi:10.1080/00207543.2020.1824083 · Cited by 34
CaSiNo: A Corpus of Campsite Negotiation Dialogues for Automatic Negotiation Systems
Chawla, K., Ramirez, J., Clever, R., Lucas, G., May, J., & Gratch, J. S. (2021) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2021.naacl-main.254 · Cited by 57
Fine-grained legal entity annotation: A case study on the Brazilian Supreme Court
Correia, F., Almeida, A. A. A., Nunes, J. L., Santos, K. G., Hartmann, I. A., Silva, F. A., & Lopes, H. (2021) · Information Processing & Management · doi:10.1016/j.ipm.2021.102794 · Cited by 38
In-group Questions and Out-group Answers
Hong, H., Gilbert, É., Abowd, G. D., & Arriaga, R. I. (2015) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/2702123.2702402 · Cited by 34
Acquired Codes of Meaning in Data Visualization and Infographics: Beyond Perceptual Primitives
Byrne, L., Angus, D. J., & Wiles, J. (2015) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2015.2467321 · Cited by 34
Quantifying Doctors’ Argumentation in General Practice Consultation Through Content Analysis: Measurement Development and Preliminary Results
Labrie, N., & Schulz, P. J. (2014) · Argumentation · doi:10.1007/s10503-014-9331-5 · Cited by 33
Embodied Learning for Well-Being, Self-Awareness, and Stress Regulation: A Randomized Trial with Engineering Students Using a Mixed-Method Approach
Jiménez, R. M. R., Carmona, M., García-Merino, S., Ureña, G. D., & Bercial, P. J. L. (2022) · Education Sciences · doi:10.3390/educsci12020111 · Cited by 36
How can data-driven service innovation transform hybrid business offerings in global markets?
Hani, U., Akter, S., Varsha, P. S., & Efthymiou, L. (2025) · International Marketing Review · doi:10.1108/imr-01-2025-0023 · Cited by 33
Collaboration structure and knowledge diffusion in Turkish management academia
Özel, B. (2012) · Scientometrics · doi:10.1007/s11192-012-0641-9 · Cited by 33
Corporate social responsibility reporting in the telecommunications sector in Ghana
Abukari, A. J., & Abdul-Hamid, I. K. (2018) · International Journal of Corporate Social Responsibility · doi:10.1186/s40991-017-0025-9 · Cited by 33
Indigenous Student Perspectives on Support and Impediments at University
Oliver, R., Grote, E., Rochecouste, J., & Dann, T. (2015) · The Australian Journal of Indigenous Education · doi:10.1017/jie.2015.16 · Cited by 33
Young adults' motivations to feed wild birds and influences on their potential participation in citizen science: An exploratory study
Martin, V. Y., & Greig, E. I. (2019) · Biological Conservation · doi:10.1016/j.biocon.2019.05.009 · Cited by 39
Exploring the effect of Weibo opinion leaders on the dynamics of public opinion in China: A revisit of the two-step flow of communication
Su, Y. (2019) · Global Media and China · doi:10.1177/2059436419866012 · Cited by 33
Network enforcement as denunciation endorsement? A critical study on legal enforcement in social media
Kasakowskij, T., Fürst, J., Fischer, J., & Fietkiewicz, K. J. (2019) · Telematics and Informatics · doi:10.1016/j.tele.2019.101317 · Cited by 33
Delay factors in construction of healthcare infrastructure projects: a comparison amongst developing countries
Mittal, Y. K., Paul, V. K., Rostami, A., Riley, M., & Sawhney, A. (2020) · Asian Journal of Civil Engineering · doi:10.1007/s42107-020-00227-1 · Cited by 33
Eras: Improving the quality control in the annotation process for Natural Language Processing tasks
Grosman, J., Furtado, P. H. T., Rodrigues, A. M. B., Schardong, G. G., Barbosa, S. D. J., & Lopes, H. (2020) · Information Systems · doi:10.1016/j.is.2020.101553 · Cited by 33
Smart technologies in the Covid-19 crisis: Managing tourism flows and shaping visitors’ behaviour
Trunfio, M., & Pasquinelli, C. (2021) · European Journal of Tourism Research · doi:10.54055/ejtr.v29i.2437 · Cited by 33
Explainable Sentiment Analysis: A Hierarchical Transformer-Based Extractive Summarization Approach
Bacco, L., Cimino, A., Dell’Orletta⋄, F., & Merone, M. (2021) · Electronics · doi:10.3390/electronics10182195 · Cited by 34
A SMART approach to personalized care: preliminary data on how to select and sequence skills in transdiagnostic CBT
Sauer-Zavala, S. E., Southward, M. W., Stumpp, N. E., Semcho, S., Hood, C. O., Garlock, A., & Urs, A. (2022) · Cognitive Behaviour Therapy · doi:10.1080/16506073.2022.2053571 · Cited by 35
Portrayal of Interactions Between Humans and Coyotes (Canis latrans): Content Analysis of Canadian Print Media (1998-2010)
Wildlife, U., Alexander, S. M., & Quinn, M. S. (2011) · Cities and the Environment · doi:10.15365/cate.4172012 · Cited by 33
The Impact of Media Coverage and Motivation on Performance-Relevant Information
Elenbaas, M., Boomgaarden, H. G., Schuck, A. R. T., & de Vreese, C. H. (2013) · Political Communication · doi:10.1080/10584609.2012.737411 · Cited by 33
Reliability from alpha to omega: a tutorialPreprint
Revelle, W., & Condon, D. M. (2018) · doi:10.31234/osf.io/2y3w9 · Cited by 33
To Err is AI: Imperfect Interventions and Repair in a Conversational Agent Facilitating Group Chat Discussions
Do, H. J., Kong, H.-K., Tetali, P., Lee, J., & Bailey, B. P. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3579532 · Cited by 32
Constructing Care-Based Corporate Social Responsibility (CSR) Communication During the COVID-19 Pandemic: A Comparison of Fortune 500 Companies in China and the United States
Dong, C., Huang, Q., Ni, S., Zhang, B., & Chen, C. (2023) · Journal of Business Ethics · doi:10.1007/s10551-023-05531-9 · Cited by 32
Measuring the morphological characteristics of thoracolumbar fascia in ultrasound images: an inter-rater reliability study
Coninck, K. D., Hambly, K., Dickinson, J. W., & Passfield, L. (2018) · BMC Musculoskeletal Disorders · doi:10.1186/s12891-018-2088-5 · Cited by 32
Entrepreneurship in the Policy Process: Linking Behavior and Context through a Systematic Review of the Policy Entrepreneurship Literature
Aviram, N. F., Beeri, I., & Cohen, N. (2019) · Public Administration Review · doi:10.1111/puar.13089 · Cited by 32
Looking for a Needle in a Haystack: How to Search for Bottom‐Up Social Innovations that Solve Complex Humanitarian Problems
Kruse, D. J., Goeldner, M., Eling, K., & Herstatt, C. (2019) · Journal of Product Innovation Management · doi:10.1111/jpim.12507 · Cited by 34
Intrinsic group behaviour II: On the dependence of triad spatial dynamics on social and personal features; and on the effect of social interaction on small group dynamics
Zanlungo, F., Yücel, Z., & Kanda, T. (2019) · PLoS ONE · doi:10.1371/journal.pone.0225704 · Cited by 32
Development of a conceptual framework for a group-based format of the Lifestyle-integrated Functional Exercise (gLiFE) programme and its initial feasibility testing
Kramer-Gmeiner, F., Labudek, S. E., Jansen, C.-P., Nerz, C., Fleig, L., Clemson, L. M., Becker, C., & Schwenk, M. (2020) · Pilot and Feasibility Studies · doi:10.1186/s40814-019-0539-x · Cited by 34
Conceptual connections between science and engineering in elementary teachers’ unit plans
Pleasants, J., Tank, K., & Olson, J. (2021) · International Journal of STEM Education · doi:10.1186/s40594-021-00274-3 · Cited by 27
Youth-appealing features in popular e-cigarette brand advertising in the USA after heightened scrutiny in 2018
Liu, J., Vázquez‐Otero, C., Berman, M. L., & Stevens, E. M. (2021) · Tobacco Control · doi:10.1136/tobaccocontrol-2021-056720 · Cited by 32

Authors: Krippendorff, K.
Corresponding author: Krippendorff, K.
ORCID: Krippendorff, K.
Venue: Human Communication Research
Publisher: Oxford University Press (OUP)
Published: 2004-07-01
Volume: 30
Issue: 3
Pages: 411-433
ISSN: 0360-3989 (ISSN-L)
1468-2958
DOI: 10.1093/hcr/30.3.411
OpenAlex: W4243989635
Cited by: 1,027
References: 0
Citations by year: 2026: 42 · 2025: 100 · 2024: 108 · 2023: 108 · 2022: 124 · 2021: 135 · 2020: 105 · 2019: 57 · 2018: 44 · 2017: 29 · 2016: 41 · 2015: 25 · 2014: 26 · 2013: 21 · 2012: 15
FWCI: 5.93
Type: journal article
Language: en
Open access: green
repository.upenn.edu
Licence: doi.wiley.com/10.1002/tdm_license_1 (tdm, 4079 d)
Locations: Human Communication Research (publishedVersion)
ScholarlyCommons (University of Pennsylvania) (submittedVersion)
Topics: Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Reliability and Agreement in Measurement
Social Sciences › General Social Sciences › Computational and Text Analysis Methods
Social Sciences › Decision Sciences › Management Science and Operations Research › Psychometric Methodologies and Testing
Keywords: inter-rater reliability, reliability coefficients, interobserver agreement, reporting reliability

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Reliability in content analysis: some common misconceptions and recommendations.

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Self-check

Citations

Cites

Cited by

Related

Metadata

Record

About these checks