The DOI matches its Crossref record; 5 of the 5 references it cites were found in a registry or an index.
Reliability in content analysis: some common misconceptions and recommendations.
Open accessAbstract
In a recent article published in this journal, Lombard, Snyder-Duch, and Bracken (2002) surveyed 200 content analyses for their reporting of reliability tests; compared the virtues and drawbacks of five popular reliability measures; and proposed guidelines and standards for their use. Their discussion revealed that numerous misconceptions circulate in the content analysis literature regarding how these measures behave and can aid or deceive content analysts in their effort to ensure the reliability of their data. This paper proposes three conditions for statistical measures to serve as indices of the reliability of data and examines the mathematical structure and the behavior of the five coefficients discussed by the authors, plus two others. It compares common beliefs about these coefficients with what they actually do and concludes with alternative recommendations for testing reliability in content analysis and similar data-making efforts.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“For the 6 moderator variables, Krippendorff’s alpha ranged from 0.81 to 0.99, and therefore above the acceptable level (Krippendorff, 2004).”
From The Power of Feedback Revisited: A Meta-Analysis of Educational Feedback Research (2020) · cites this paper as Krippendorff, 2004
The cited source says what the claim attributes to it.
In the citing paper
“Krippendorff's (2004) criterion of significance is therefore satisfied here too.”
From Weighted Krippendorff's alpha is a more reliable metrics for multi-coders ordinal annotations: experimental studies on emotion, opinion and coreference annotation (2014) · cites this paper as Krippendorff's (2004)
The cited source says what the claim attributes to it.
In the citing paper
“Krippendorff's α α α α (Krippendorff, 2004) considers chance independently from coders like Scott's π, but data reliability is estimated depending on disagreement instead of agreement:”
From Weighted Krippendorff's alpha is a more reliable metrics for multi-coders ordinal annotations: experimental studies on emotion, opinion and coreference annotation (2014) · cites this paper as Krippendorff, 2004
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“The criterion of data significance -at least five chance agreements per category -proposed by (Krippendorff, 2004) is greatly satisfied for the valence annotation (3 categories).”
From Weighted Krippendorff's alpha is a more reliable metrics for multi-coders ordinal annotations: experimental studies on emotion, opinion and coreference annotation (2014) · cites this paper as Krippendorff, 2004
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The publisher hasn’t deposited this paper’s reference list.
Research works printed but not linked: 13
- Bennett, E. M., Alpert, R., & Goldstein, A. C. (1954). Communications through limited response questioning. Public Opinion Quarterly, 18, 303-308.
- Craig, R. T. (1981). Generalization of Scott"s index of intercoder agreement. Public Opinion Quarterly, 45, 260-264.
- Goodman, L. A, & Kruskal, W. H. (1954). Measures of association for cross classifications. Journal of the American Statistical Association, 49, 732-764.
- Holsti, O. R. (1969). Content analysis for the social sciences and humanities. Reading, MA: Addison- Wesley.
- Hughes, M. A., & Garrett, D. E. (1990). Intercoder reliability estimation -Approaches in marketing: A generalizability theory framework for quantitative data. Journal of Marketing Research, 27, 185- 195.
- Krippendorff, K. (1970). Bivariate agreement coefficients for reliability data. In E. R. Borgatta & G. W. Bohrnstedt (Eds.), Sociological methodology 1970 (pp. 139-150). San Francisco, CA: Jossey Bass.
- Krippendorff, K. (2004). Content analysis: An introduction to its methodology. Second Edition. Thousand Oaks, CA: Sage.
- Maxwell, A. E. (1970). Comparing the classification of subjects by two independent judges. British Journal of Psychiatry, 116, 651-655.
- Osgood, C. E. (1959). The representational model and relevant research. In I. de Sola Pool (Ed.), Trends in content analysis (pp. 33-88). Urbana: University of Illinois Press.
- Potter, W. J., & Levine-Donnerstein, D. (1999). Rethinking validity and reliability in content analysis. Journal of Applied Communication Research, 27, 258-284.
- Riffe, D., & Freitag, A. (1996, August). Twenty-five years of content analyses in Journalism & Mass Communication Quarterly. Paper presented to the annual convention of the Association for Education in Journalism and Mass Communication, Anaheim, CA, cited in D. Riffe, S. Lacy, & F. G. Fico (1998). Analyzing media messages. Mahwah, NJ: Erlbaum.
- Scott, W. A. (1955). Reliability of content analysis: The case of nominal scale coding. Public Opinion Quarterly, 19, 321-325.
- Zwick, R. (1988). Another look at interrater agreement. Psychological Bulletin, 103, 347-387.
Cited by
Sorted by citations
Items 701–800 of 1,010
Assessing the reliability of student evaluations of teaching: choosing the right coefficient
Morley, D. D. (2013) · Assessment & Evaluation in Higher Education · doi:10.1080/02602938.2013.796508 · Cited by 31
Nuevas tendencias de la extensión rural para el desarrollo de capacidades de autogestión
Espinosa, H. R., Gómez, C. J. R., & Betancur, L. F. R. (2016) · Ciencia y Tecnología Agropecuaria · doi:10.21930/rcta.vol17_num1_art:457 · Cited by 28
Twenty Years of Social Studies Textbook Content Analysis: Still “Decidedly Disappointing”?
Chu, Y. (2017) · The Social Studies · doi:10.1080/00377996.2017.1360240 · Cited by 28
Yu, K., Cadeaux, J. M., Luo, N., Qian, C., & Chen, Z. (2018) · Industrial Management & Data Systems · doi:10.1108/imds-09-2017-0410 · Cited by 28
Social Media for Informal Learning: a Case of #Twitterstorians
Kumar, P. C., & Gruzd, A. A. (2019) · Hawaii International Conference on System Sciences (HICSS) · doi:10.24251/hicss.2019.304 · Cited by 28
The Sinclair Effect: Comparing Ownership Influences on Bias in Local TV News Content
Hedding, K. J., Miller, K. C., Abdenour, J., & Blankenship, J. C. (2019) · Journal of Broadcasting & Electronic Media · doi:10.1080/08838151.2019.1653103 · Cited by 28
Goyanes, M., & Piñeiro-Naval, V. (2024) · Estudios sobre el Mensaje Periodístico · doi:10.5209/esmp.92732 · Cited by 27
Characteristics of innovation in adopting a renewable residential energy system
Tapaninen, A., Seppänen, M., & Mäkinen, S. (2009) · Journal of Systems and Information Technology · doi:10.1108/13287260911002495 · Cited by 29
Stereotype threat and gender differences in chemistry
Sunny, C. E., Taasoobshirazi, G., Clark, L. B., & Marchand, G. C. (2016) · Instructional Science · doi:10.1007/s11251-016-9395-8 · Cited by 29
Bijl, A., Ahaus, K., Ruël, G. C., Gemmel, P., & Meijboom, B. R. (2019) · BMJ Open · doi:10.1136/bmjopen-2018-026737 · Cited by 27
Sustainability in the wine sector
Mastroberardino, P., Calabrese, G., Cortese, F., & Petracca, M. (2019) · British Food Journal · doi:10.1108/bfj-07-2019-0475 · Cited by 27
De-Westernizing Media Parallelism: How Editorial Interests Unfold During Impeachment Crises
Pimentel, P. S., & Marques, F. P. J. (2021) · Journalism Studies · doi:10.1080/1461670x.2020.1867000 · Cited by 27
McGloin, R. S. (2021) · Teaching in Higher Education · doi:10.1080/13562517.2021.1898364 · Cited by 27
Curating a Culture: The Portrayal of Disability Stereotypes by Paralympians on Instagram
Mitchell, F. R., van Wyk, P. M., & Santarossa, S. (2021) · International Journal of Sport Communication · doi:10.1123/ijsc.2021-0030 · Cited by 27
Forest Value Orientations in Australia: An Application of Computer Content Analysis
Webb, T. J., Βengston, D. Ν., & Fan, D. P. (2007) · Environmental Management · doi:10.1007/s00267-007-9011-4 · Cited by 27
Gartrell, N., Bos, H., Goldberg, N. G., Deck, A., & Gelderen, L. V. R. (2014) · Fertility and Sterility · doi:10.1016/j.fertnstert.2014.09.019 · Cited by 27
Weighted kappa is higher than Cohen’s kappa for tridiagonal agreement tables
Warrens, M. J. (2010) · Statistical Methodology · doi:10.1016/j.stamet.2010.09.004 · Cited by 27
Underlying determinants driving agreement among coders
Feng, G. C. (2012) · Quality & Quantity · doi:10.1007/s11135-012-9807-z · Cited by 31
Matney, S. A., Maddox, L. J., & Staggers, N. (2013) · Western Journal of Nursing Research · doi:10.1177/0193945913497111 · Cited by 27
Inquiry-based Instruction with Archived, Online Data: An Intervention Study with Preservice Teachers
Uçar, S., Trundle, K. C., & Krissek, L. A. (2010) · Research in Science Education · doi:10.1007/s11165-009-9164-7 · Cited by 27
Xipho: extending tropos to engineer context-aware personal agents
Murukannaiah, P. K., & Singh, M. P. (2014) · doi:10.5555/2615731.2615783 · Cited by 27
Boonen, A. J. H., Reed, H. C., Schoonenboom, J., & Jolles, J. (2016) · Frontline Learning Research · doi:10.14786/flr.v4i5.245 · Cited by 27
Behavioral Engagement Detection of Students in the Wild
Okur, E., Alyüz, N., Aslan, S., Genc, U., Tanriover, C., & Esme, A. A. (2017) · Lecture notes in computer science · doi:10.1007/978-3-319-61425-0_21 · Cited by 27
Agreement is overrated: A plea for correlation to assess human evaluation reliability
Amidei, J., Piwek, P., & Willis, A. (2019) · International Conference on Natural Language Generation · doi:10.18653/v1/w19-8642 · Cited by 27
Authentic Self and Last Resort: International Perceptions of Psychiatric Neurosurgery
Cabrera, L. Y., Courchesne, C., Bittlinger, M., Müller, S., Martínez, R., Racine, É., & Illes, J. (2020) · Culture Medicine and Psychiatry · doi:10.1007/s11013-020-09679-1 · Cited by 28
Scalone, L., Tomasetto, C., Matteucci, M. C., Selleri, P., Broccoli, S., Pacelli, B., & Cavrini, G. (2011) · Italian Journal of Public Health · doi:10.2427/5679 · Cited by 27
Methodological Rigor in Quantitative Research
Marquart, F. (2017) · The International Encyclopedia of Communication Research Methods · doi:10.1002/9781118901731.iecrm0221 · Cited by 27
Manika, D., Iacovidou, E., Canhoto, A. I., Pei, E., & Mach, K. (2022) · Journal of Cleaner Production · doi:10.1016/j.jclepro.2022.133449 · Cited by 26
Decoding the Digital Campus Climate for Prospective LGBTQ+ Community Colleges Students
Taylor, J. L., Dockendorff, K. J., & Inselman, K. M. (2017) · Community College Journal of Research and Practice · doi:10.1080/10668926.2017.1281177 · Cited by 26
Meier, J. G., Wu, C. C., Cuellar, S. L. B., Truong, M. T., Erasmus, J. J., Einstein, S. A., & Mawlawi, O. R. (2018) · Journal of Nuclear Medicine · doi:10.2967/jnumed.118.213884 · Cited by 26
Joe, S., Choi, C., & Busser, J. A. (2021) · International Journal of Hospitality Management · doi:10.1016/j.ijhm.2021.103021 · Cited by 26
G20 summit menus as a means of gastrodiplomacy: Messages to the world via menus
Demir, M., & Alper, P. (2021) · International Journal of Gastronomy and Food Science · doi:10.1016/j.ijgfs.2021.100368 · Cited by 27
The Effects of Three Instructor Participatory Roles on a Small Group’s Collaborative Concept Mapping
Ouyang, F., & Xu, W. (2021) · Journal of Educational Computing Research · doi:10.1177/07356331211057283 · Cited by 27
Female, white, 27? Bias Evaluation on Data and Algorithms for Affect Recognition in Faces
Pahl, J., Rieger, I., Möller, A., Wittenberg, T. C., & Schmid, U. (2022) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3531146.3533159 · Cited by 26
Miller, A. N., Deeter, C., Trelstad, A., Hawk, M., Ingram, G., & Ramirez, A. (2013) · Journal of International and Intercultural Communication · doi:10.1080/17513057.2013.787112 · Cited by 26
Using Qualitative Methods for Revising Items in the Hispanic Stress Inventory
Cervantes, R. C., Goldbach, J. T., & Padilla, A. M. (2012) · Hispanic Journal of Behavioral Sciences · doi:10.1177/0739986312442495 · Cited by 26
Message Expression Effects in Online Social Communication
Nekmat, E. (2012) · Journal of Broadcasting & Electronic Media · doi:10.1080/08838151.2012.678513 · Cited by 26
Perko, T., Turcanu, C., & Carlé, B. (2012) · Journal of Contingencies and Crisis Management · doi:10.1111/j.1468-5973.2012.00663.x · Cited by 26
Guiding teaching with assessments: high school chemistry teachers’ use of data-driven inquiry
Harshman, J. T., & Yezierski, E. J. (2014) · Chemistry Education Research and Practice · doi:10.1039/c4rp00188e · Cited by 26
McGivney, C. L., Sweeney, J. S., David, F., O'Leary, J. M., Hill, E. W., & Katz, L. M. (2016) · Equine Veterinary Journal · doi:10.1111/evj.12653 · Cited by 26
Scholman, M., & Demberg, V. (2017) · Linguistic Annotation Workshop · doi:10.18653/v1/w17-0803 · Cited by 26
Measuring Extremes: A Quantitative Content Analysis of Prime Time TV Depictions of Body Type
Mastro, D. E., & Figueroa-Caballero, A. (2018) · Journal of Broadcasting & Electronic Media · doi:10.1080/08838151.2018.1451853 · Cited by 26
Nagler, R. H., Fowler, E. F., Marino, N. M., Mentzer, K. M., & Gollust, S. E. (2018) · Women s Health Issues · doi:10.1016/j.whi.2018.09.005 · Cited by 26
A Possible Synergy between Culture and Religion for the Sustainability of Tourism of Pompeii
Cortese, F., D’Ambrosio, I., & Petracca, M. (2019) · Sustainability · doi:10.3390/su11082231 · Cited by 26
Carle, M. S., Visser, R., & Flynn, A. B. (2020) · Chemistry Education Research and Practice · doi:10.1039/c9rp00274j · Cited by 26
Jafree, S. R., Zakar, R., Rafiq, N., Javed, A., Durrani, R. R., Burhan, S. K., Nadir, S. M. H., Ali, F. Z., Shahid, A., Momina, A. U., et al. (2022) · Frontiers in Public Health · doi:10.3389/fpubh.2022.739761 · Cited by 27
Learn to Lead: Mapping Workplace Learning of School Leaders
Hulsbos, F., Evers, A. T., & Kessels, J. (2015) · Vocations and Learning · doi:10.1007/s12186-015-9140-5 · Cited by 26
To whom it may concern: addressee identification in face-to-face meetings
Jovanović, N. (2007) · doi:10.3990/1.9789036524704 · Cited by 26
Election Campaigns and Welfare State Change
Kumlin, S., & Goerres, A. (2022) · doi:10.1093/oso/9780198869214.001.0001 · Cited by 25
Watchorn, V., Hitch, D., Tucker, R., Frawley, P., Aedy, K., & Grant, C. (2022) · Journal of Housing and the Built Environment · doi:10.1007/s10901-022-09981-x · Cited by 25
Accounting for human rights: Evidence of due diligence in EU-listed firms’ reporting
Rogerson, M., Scarpa, F., & Snelson-Powell, A. C. (2024) · Critical Perspectives on Accounting · doi:10.1016/j.cpa.2024.102716 · Cited by 25
Innovation at the interface: A configurational approach to corporate venture capital
Schückes, M., Unger, B., Gutmann, T., & Fels, G. (2024) · Journal of Business Venturing · doi:10.1016/j.jbusvent.2024.106438 · Cited by 28
Systemic Risk Management and Stakeholder Engagement: Insights From Business CSR Disclosure
Battaglia, M., Ceglia, I., Calabrese, M., & Iandolo, F. (2025) · Corporate Social Responsibility and Environmental Management · doi:10.1002/csr.3186 · Cited by 25
Rumormongering of genetically modified (GM) food on Chinese social network
Ji, J., Chao, N.-P., & Ding, J. (2019) · Telematics and informatics · doi:10.1016/j.tele.2019.01.005 · Cited by 27
Müller-Budack, E., Theiner, J., Rein, R., & Ewerth, R. (2019) · ACM International Workshop on Multimedia Content Analysis in Sports (MMSports) · doi:10.1145/3347318.3355527 · Cited by 25
Truelove, M. A., Martin, J., Langford, F. M., & Leach, M. C. (2020) · Scientific Reports · doi:10.1038/s41598-020-77437-9 · Cited by 26
Reasoning in social media: insights from Reddit “Change My View” submissions
Öcal, A., Xiao, L., & Park, J. (2021) · Online Information Review · doi:10.1108/oir-08-2020-0330 · Cited by 25
Modeling Educational Discourse with Natural Language Processing
Dowell, N., & Kovanović, V. (2022) · Solar eBooks · doi:10.18608/hla22.011 · Cited by 25
ToxiSpanSE: An Explainable Toxicity Detection in Code Review Comments
Sarker, J., Sultana, S., Wilson, S. R., & Bosu, A. (2023) · Proceedings of the ... ACM-IEEE International Symposium on Empirical Software Engineering and Measurement/International Symposium on Empirical Software Engineering and Measurement · doi:10.1109/esem56168.2023.10304855 · Cited by 31
Willoughby, J. F., & Jackson, K. (2012) · Sex Education · doi:10.1080/14681811.2012.677746 · Cited by 25
Factors impacting the quality of peer relationships of youth with Tourette’s syndrome
O'hare, D., Eapen, V., Helmes, E., McBain, K., Reece, J., & Grove, R. (2015) · BMC Psychology · doi:10.1186/s40359-015-0090-3 · Cited by 28
Examining Public Response to Corporate Social Initiative Types
Austin, L., & Gaither, B. M. (2016) · Social Marketing Quarterly · doi:10.1177/1524500416642441 · Cited by 25
O’Brien, K. M., & Daffern, M. (2016) · Psychiatry Psychology and Law · doi:10.1080/13218719.2016.1209804 · Cited by 25
Benefit-based O2O commerce segmentation: a means-end chain approach
Xiao, L., Guo, Z., & D’Ambra, J. D. (2018) · Electronic Commerce Research · doi:10.1007/s10660-017-9286-3 · Cited by 25
Policy entrepreneurship in developing countries: A systematic review of the literature
Aviram, N. F., Cohen, N., & Beeri, I. (2019) · Public Administration and Development · doi:10.1002/pad.1852 · Cited by 25
Ataei, P., Sadighi, H., Chizari, M., & Abbasi, E. (2019) · Environment Development and Sustainability · doi:10.1007/s10668-019-00484-4 · Cited by 25
Pleasants, J., Olson, J. K., & Cruz, I. D. L. (2020) · Journal of Science Teacher Education · doi:10.1080/1046560x.2019.1709295 · Cited by 25
Petruzzi, M. A., Marques, C. M. V. A., & Sheppard, V. A. (2020) · International Journal of Hospitality Management · doi:10.1016/j.ijhm.2020.102724 · Cited by 27
Shi, D. (2020) · International Journal of Strategic Communication · doi:10.1080/1553118x.2020.1835920 · Cited by 25
Looking to the Middle of the Qualitative-Quantitative Spectrum for Integrated Mixed Methods
Paoletti, J., Bisbey, T. M., Zajac, S. A., Waller, M. J., & Salas, E. (2021) · Small Group Research · doi:10.1177/1046496421992433 · Cited by 38
Keaton, S. A., & Gearhart, C. C. (2013) · Communication & Sport · doi:10.1177/2167479513498077 · Cited by 26
Towards interactive event log forensics: Detecting and quantifying timestamp imperfections
Fischer, D. A., Goel, K., Andrews, R., van Dun, C., Wynn, M. T., & Röglinger, M. (2022) · Information Systems · doi:10.1016/j.is.2022.102039 · Cited by 28
The drivers of nonfinancial disclosure quality: A systematic literature review analysis
Latella, P., & Veltri, S. (2024) · Corporate Social Responsibility and Environmental Management · doi:10.1002/csr.2880 · Cited by 24
Delgado-Galindo, P., García-Jiménez, J., Gordillo, J. J. T., & Rodríguez-Santero, J. (2025) · Sustainability · doi:10.3390/su17146497 · Cited by 30
Bockstal, M. V., Baldewijns, M. M. L. L., Colpaert, C., Dano, H., Floris, G., Galant, C., Lambein, K., Peeters, D. J. E., Renterghem, S. V., Rompuy, A. V., et al. (2018) · Histopathology · doi:10.1111/his.13741 · Cited by 24
Comparing the Utility of Different Classification Schemes for Emotive Language Analysis
Williams, L., Arribas‐Ayllon, M., Artemiou, A., & Spasić, . (2019) · Journal of Classification · doi:10.1007/s00357-019-9307-0 · Cited by 24
Cybersecurity and Digital Exclusion of Seniors: What Do They Fear?
Holgersson, J., Kävrestad, J., & Nohlberg, M. (2021) · IFIP advances in information and communication technology · doi:10.1007/978-3-030-81111-2_2 · Cited by 24
The Modern Character of Mental Health Stigma: A 30-Year Examination of Popular Film
Riles, J. M., Miller, B. M., Funk, M. E., & Morrow, E. (2021) · Communication Studies · doi:10.1080/10510974.2021.1953098 · Cited by 29
SaFeRDialogues: Taking Feedback Gracefully after Conversational Safety Failures
Ung, M., Xu, J., & Boureau, Y.-L. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.447 · Cited by 31
Yerger, J. C., Cougnoux, A., Abbott, C. B., Luke, R., Clark, T. S., Cawley, N. X., Porter, F. D., & Davidson, C. D. (2022) · Biology Open · doi:10.1242/bio.059052 · Cited by 24
Investigating Influencer VPN Ads on YouTube
Akgul, O., Roberts, R. M., Namara, M., Levin, D., & Mazurek, M. L. (2022) · IEEE Symposium on Security and Privacy (S&P) · doi:10.1109/sp46214.2022.9833633 · Cited by 25
Harris, H. A., & Brison, N. T. (2022) · International Journal of Sport Communication · doi:10.1123/ijsc.2022-0113 · Cited by 24
Emerging HR analytics role in a crisis: an analysis of LinkedIn data
Kashive, N., & Khanna, V. T. (2022) · Competitiveness Review An International Business Journal incorporating Journal of Global Competitiveness · doi:10.1108/cr-03-2022-0029 · Cited by 30
Examining Implicit Biases of Pre-Service Educators Within a Professional Development Context
Breese, A. C., Nickerson, A. B., Lemke, M., Mohr, R., Heidelburg, K., Fredrick, S. S., & Allen, K. E. (2023) · Contemporary School Psychology · doi:10.1007/s40688-023-00456-6 · Cited by 32
Do Large Language Models Pay Similar Attention Like Human Programmers When Generating Code?
Kou, B., Chen, S., Wang, Z., Ma, L., & Zhang, T. (2024) · Proceedings of the ACM on software engineering. · doi:10.1145/3660807 · Cited by 42
Evidence-based instruction integration: a syllabus analysis project
Boss, K. E., & Drabinski, E. (2014) · Reference Services Review · doi:10.1108/rsr-07-2013-0038 · Cited by 24
Evaluating Feature Sets and Classifiers for Sentiment Analysis of Financial News
Njølstad, P.-C. S., Høysæter, L. S., Wei, W., & Gulla, J. A. (2014) · IEEE/WIC/ACM International Conference on Web Intelligence and Intelligent Agent Technology (WI-IAT) · doi:10.1109/wi-iat.2014.82 · Cited by 24
Inconsistency in the analysis of morphological deformities in chironomidae (Insecta: Diptera) larvae
Salmelin, J., Vuori, K.-M., & Hämäläinen, H. (2015) · Environmental Toxicology and Chemistry · doi:10.1002/etc.3010 · Cited by 24
Galbraith, S. M., Hall, T. E., Tavárez, H. S., Kooistra, C. M., Ordoñez, J. C., & Bosque‐Pérez, N. A. (2017) · Land Use Policy · doi:10.1016/j.landusepol.2017.08.032 · Cited by 27
Off the charts: user engagement enhancers in election infographics
Amit-Danhi, E. R., & Shifman, L. (2020) · Information Communication & Society · doi:10.1080/1369118x.2020.1761858 · Cited by 24
Nudging is Ineffective When Attitudes Are Unsupportive: An Example from a Natural Field Experiment
Dewies, M., Schop-Etman, A., Rohde, K. I. M., & Denktaş, S. (2021) · Basic and Applied Social Psychology · doi:10.1080/01973533.2021.1917412 · Cited by 24
Risk Disclosure and Corporate Cash Holdings
Haj-Salem, I., & Hussainey, K. (2021) · Journal of risk and financial management · doi:10.3390/jrfm14070328 · Cited by 24
Månsson, P., Abrahamsson, M., Hassel, H., & Tehler, H. (2015) · International Journal of Disaster Risk Reduction · doi:10.1016/j.ijdrr.2015.08.003 · Cited by 24
How does real affect affect affect recognition in speech?
Truong, K. P. (2009) · doi:10.3990/1.9789036528801 · Cited by 24
Correspondence Analysis: A New Method for Analyzing Qualitative Data in Architecture
Habib, F., Etesam, I., Ghoddusifar, S. H., & Mohajeri, N. (2012) · Springer eBooks · doi:10.1007/s00004-012-0129-1 · Cited by 24
Evidence-based programming language design : a philosophical and methodological exploration
Kaijanaho, A.-J. (2015) · Jyväskylä University Digital Archive (University of Jyväskylä) · Cited by 24
Slim, T., van Schaik, J. E., Dobber, M., Hotze, A. C. G., & Raijmakers, M. E. J. (2022) · Frontiers in Education · doi:10.3389/feduc.2022.842537 · Cited by 23
Yadav, H., Kar, A. K., Kashiramka, S., & Rana, N. P. (2024) · Technological Forecasting and Social Change · doi:10.1016/j.techfore.2024.123649 · Cited by 23
Parviz, M. (2024) · Estudios Ingleses de la Universidad Complutense · doi:10.5209/cjes.97261 · Cited by 29
Signs of Europeanization?: the 2014 EP election in European newspapers
Belluati, M. (2016) · Italian Political Science Review/Rivista Italiana di Scienza Politica · doi:10.1017/ipo.2016.10 · Cited by 23
Metadata
- Authors
- Krippendorff, K.
- Corresponding author
- Krippendorff, K.
- ORCID
- Krippendorff, K.
- Venue
- Human Communication Research
- Publisher
- Oxford University Press (OUP)
- Published
- 2004-07-01
- Volume
- 30
- Issue
- 3
- Pages
- 411-433
- ISSN
- 0360-3989 (ISSN-L)
- 1468-2958
- OpenAlex
- W4243989635
- Cited by
- 1,027
- References
- 0
- Citations by year
- 2026: 42 · 2025: 100 · 2024: 108 · 2023: 108 · 2022: 124 · 2021: 135 · 2020: 105 · 2019: 57 · 2018: 44 · 2017: 29 · 2016: 41 · 2015: 25 · 2014: 26 · 2013: 21 · 2012: 15
- FWCI
- 5.93
- Type
- journal article
- Language
- en
- Open access
- green
- repository.upenn.edu
- Locations
- Human Communication Research (publishedVersion)
- ScholarlyCommons (University of Pennsylvania) (submittedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Reliability and Agreement in Measurement
- Social Sciences › General Social Sciences › Computational and Text Analysis Methods
- Social Sciences › Decision Sciences › Management Science and Operations Research › Psychometric Methodologies and Testing
- Keywords
- inter-rater reliability, reliability coefficients, interobserver agreement, reporting reliability
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
