The DOI matches its Crossref record; 5 of the 5 references it cites were found in a registry or an index.
Reliability in content analysis: some common misconceptions and recommendations.
Open accessAbstract
In a recent article published in this journal, Lombard, Snyder-Duch, and Bracken (2002) surveyed 200 content analyses for their reporting of reliability tests; compared the virtues and drawbacks of five popular reliability measures; and proposed guidelines and standards for their use. Their discussion revealed that numerous misconceptions circulate in the content analysis literature regarding how these measures behave and can aid or deceive content analysts in their effort to ensure the reliability of their data. This paper proposes three conditions for statistical measures to serve as indices of the reliability of data and examines the mathematical structure and the behavior of the five coefficients discussed by the authors, plus two others. It compares common beliefs about these coefficients with what they actually do and concludes with alternative recommendations for testing reliability in content analysis and similar data-making efforts.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“For the 6 moderator variables, Krippendorff’s alpha ranged from 0.81 to 0.99, and therefore above the acceptable level (Krippendorff, 2004).”
From The Power of Feedback Revisited: A Meta-Analysis of Educational Feedback Research (2020) · cites this paper as Krippendorff, 2004
The cited source says what the claim attributes to it.
In the citing paper
“Krippendorff's (2004) criterion of significance is therefore satisfied here too.”
From Weighted Krippendorff's alpha is a more reliable metrics for multi-coders ordinal annotations: experimental studies on emotion, opinion and coreference annotation (2014) · cites this paper as Krippendorff's (2004)
The cited source says what the claim attributes to it.
In the citing paper
“Krippendorff's α α α α (Krippendorff, 2004) considers chance independently from coders like Scott's π, but data reliability is estimated depending on disagreement instead of agreement:”
From Weighted Krippendorff's alpha is a more reliable metrics for multi-coders ordinal annotations: experimental studies on emotion, opinion and coreference annotation (2014) · cites this paper as Krippendorff, 2004
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“The criterion of data significance -at least five chance agreements per category -proposed by (Krippendorff, 2004) is greatly satisfied for the valence annotation (3 categories).”
From Weighted Krippendorff's alpha is a more reliable metrics for multi-coders ordinal annotations: experimental studies on emotion, opinion and coreference annotation (2014) · cites this paper as Krippendorff, 2004
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The publisher hasn’t deposited this paper’s reference list.
Research works printed but not linked: 13
- Bennett, E. M., Alpert, R., & Goldstein, A. C. (1954). Communications through limited response questioning. Public Opinion Quarterly, 18, 303-308.
- Craig, R. T. (1981). Generalization of Scott"s index of intercoder agreement. Public Opinion Quarterly, 45, 260-264.
- Goodman, L. A, & Kruskal, W. H. (1954). Measures of association for cross classifications. Journal of the American Statistical Association, 49, 732-764.
- Holsti, O. R. (1969). Content analysis for the social sciences and humanities. Reading, MA: Addison- Wesley.
- Hughes, M. A., & Garrett, D. E. (1990). Intercoder reliability estimation -Approaches in marketing: A generalizability theory framework for quantitative data. Journal of Marketing Research, 27, 185- 195.
- Krippendorff, K. (1970). Bivariate agreement coefficients for reliability data. In E. R. Borgatta & G. W. Bohrnstedt (Eds.), Sociological methodology 1970 (pp. 139-150). San Francisco, CA: Jossey Bass.
- Krippendorff, K. (2004). Content analysis: An introduction to its methodology. Second Edition. Thousand Oaks, CA: Sage.
- Maxwell, A. E. (1970). Comparing the classification of subjects by two independent judges. British Journal of Psychiatry, 116, 651-655.
- Osgood, C. E. (1959). The representational model and relevant research. In I. de Sola Pool (Ed.), Trends in content analysis (pp. 33-88). Urbana: University of Illinois Press.
- Potter, W. J., & Levine-Donnerstein, D. (1999). Rethinking validity and reliability in content analysis. Journal of Applied Communication Research, 27, 258-284.
- Riffe, D., & Freitag, A. (1996, August). Twenty-five years of content analyses in Journalism & Mass Communication Quarterly. Paper presented to the annual convention of the Association for Education in Journalism and Mass Communication, Anaheim, CA, cited in D. Riffe, S. Lacy, & F. G. Fico (1998). Analyzing media messages. Mahwah, NJ: Erlbaum.
- Scott, W. A. (1955). Reliability of content analysis: The case of nominal scale coding. Public Opinion Quarterly, 19, 321-325.
- Zwick, R. (1988). Another look at interrater agreement. Psychological Bulletin, 103, 347-387.
Cited by
Sorted by citations
Items 601–700 of 1,010
Fostering Equality in Education: The Blockchain Business Model for Higher Education (BBM-HE)
Son-Turan, S. (2022) · Sustainability · doi:10.3390/su14052955 · Cited by 32
A data multiverse analysis investigating non‐model based SCR quantification approaches
Sjouwerman, R., Illius, S., Kuhn, M., & Lonsdorf, T. B. (2022) · Psychophysiology · doi:10.1111/psyp.14130 · Cited by 34
Using Adaptive Parsons Problems to Scaffold Write-Code Problems
Hou, X., Ericson, B. J., & Wang, X. (2022) · ACM Conference on International Computing Education Research (ICER) · doi:10.1145/3501385.3543977 · Cited by 33
Calvo, N., Amor, A. M., & Larrínaga, Ó. V. (2022) · Technological Forecasting and Social Change · doi:10.1016/j.techfore.2022.122069 · Cited by 32
Dratsch, T., Siedek, F., Zäske, C., Sonnabend, K., Rauen, P., Terzis, R., Hahnfeldt, R., Maintz, D. C., ten Persigehl, T., Bratke, G., et al. (2023) · European Radiology Experimental · doi:10.1186/s41747-023-00377-2 · Cited by 32
Blades, J. J., Klos, P. Z., Kemp, K. B., Hall, T. E., Force, J. E., Morgan, P., & Tinkham, W. T. (2015) · Forest Ecology and Management · doi:10.1016/j.foreco.2015.07.020 · Cited by 32
A Prototype of Multicomponent Brand Personality Structure: A Consumption Synmbolism Approach
Lee, E. (2013) · Psychology and Marketing · doi:10.1002/mar.20596 · Cited by 32
Markward, M. J., Benner, K., & Freese, R. (2013) · Families Systems & Health · doi:10.1037/a0034440 · Cited by 32
Cohen’s Linearly Weighted Kappa is a Weighted Average of 2×2 Kappas
Warrens, M. J. (2011) · Psychometrika · doi:10.1007/s11336-011-9210-z · Cited by 32
Do open‐ended survey questions on migration motives create coder variability problems?
Niedomysl, T., & Malmberg, B. (2008) · Population Space and Place · doi:10.1002/psp.493 · Cited by 32
Cohen’s kappa can always be increased and decreased by combining categories
Warrens, M. J. (2010) · Statistical Methodology · doi:10.1016/j.stamet.2010.05.003 · Cited by 32
Proksch, D., Stranz, W., Röhr, N., Ernst, C., Pinkwart, A., & Schefczyk, M. (2016) · Venture Capital · doi:10.1080/13691066.2016.1242573 · Cited by 32
How do STEM-interested students pursue multiple interests in their higher educational choice?
Vulperhorst, J. P., Wessels, K. R., Bakker, A., & Akkerman, S. F. (2018) · International Journal of Science Education · doi:10.1080/09500693.2018.1452306 · Cited by 32
Redmiles, E. M., Chachra, N., & Waismeyer, B. (2018) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3173786 · Cited by 32
Pollock, K. E., & Winton, S. (2015) · Educational Assessment Evaluation and Accountability · doi:10.1007/s11092-015-9224-7 · Cited by 32
Inauthentic Newsfeeds and Agenda Setting in a Coordinated Inauthentic Information Operation
Ehrett, C., Linvill, D. L., Smith, D. H., Warren, P. L., Bellamy, L., Moawad, M., Moran, O., & Moody, M. (2021) · Social Science Computer Review · doi:10.1177/08944393211019951 · Cited by 34
Do, H. J., Kong, H., Lee, J., & Bailey, B. P. (2022) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3555112 · Cited by 41
PolicyChecker: Analyzing the GDPR Completeness of Mobile Apps' Privacy Policies
Xiang, A., Pei, W., & Yue, C. (2023) · ACM SIGSAC Conference on Computer and Communications Security (CCS) · doi:10.1145/3576915.3623067 · Cited by 38
On the Effectiveness of LLM-as-a-Judge for Code Generation and Summarization
Crupi, G., Tufano, R., Velasco, A., Mastropaolo, A., Poshyvanyk, D., & Bavota, G. (2025) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2025.3586082 · Cited by 55
Relational Maintenance in the Digital Age: Implicit Rules and Multiple Modalities
Eden, J., & Veksler, A. E. (2016) · Communication Quarterly · doi:10.1080/01463373.2015.1103279 · Cited by 31
Evidence of Linguistic Intergroup Bias in U.S. Print News Coverage of Immigration
Dragojević, M., Sink, A., & Mastro, D. E. (2016) · Journal of Language and Social Psychology · doi:10.1177/0261927x16666884 · Cited by 42
Defining diabetes and assigning responsibility: how print media frame diabetes in New Zealand
Gounder, F., & Ameer, R. (2017) · Journal of Applied Communication Research · doi:10.1080/00909882.2017.1409907 · Cited by 31
Climate Hypocrisies: A Comparative Study of News Discourse
Gunster, S., Fleet, D., Paterson, M., & Saurette, P. (2018) · Environmental Communication · doi:10.1080/17524032.2018.1474784 · Cited by 31
Evolution and latest trends of local government efficiency: Worldwide research (1928–2019)
López, N. R., García, J. M., Uribe-Toril, J., & Valenciano, J. D. P. (2020) · Journal of Cleaner Production · doi:10.1016/j.jclepro.2020.121276 · Cited by 31
van Ewijk, A. R., Oikkonen, E., & Belghiti-Mahut, S. (2020) · The International Journal of Management Education · doi:10.1016/j.ijme.2020.100420 · Cited by 31
Reinterpreting governance in smart cities: An ecosystem-based view
Polese, F., Troisi, O., Grimaldi, M., & Loia, F. (2021) · Elsevier eBooks · doi:10.1016/b978-0-323-85151-0.00006-3 · Cited by 31
Polo, M. P., Hernández‐Santaolalla, V., & Lozano-González, A.-A. (2021) · El Profesional de la Informacion · doi:10.3145/epi.2021.ene.19 · Cited by 31
What does corporate social advocacy signal? Evidence from boycott participation decisions
Afego, P., & Alagidede, P. (2021) · Journal of Capital Markets Studies · doi:10.1108/jcms-10-2020-0040 · Cited by 39
David, M. M., Tataru, C. A., Daniels, J., Schwartz, J. N., Keating, J., Hampton‐Marcell, J., Gottel, N., Gilbert, J. A., & Wall, D. P. (2021) · mSystems · doi:10.1128/msystems.00193-20 · Cited by 33
A practitioner study on the implementation of strategy instruction for boredom regulation
Nakamura, S., Darasawang, P., & Reinders, H. (2021) · Language Teaching Research · doi:10.1177/13621688211010272 · Cited by 31
Evaluation of Precautionary Measures Taken for COVID-19 in the Hospitality Industry During Pandemic
Davras, Ö., & Durgun, S. (2021) · Journal of Quality Assurance in Hospitality & Tourism · doi:10.1080/1528008x.2021.1932013 · Cited by 19
Preservice Elementary Teachers Conceptions and Self-Efficacy for Integrated STEM
Menon, D., Shorman, D. A., Cox, D., & Thomas, A. K. (2023) · Education Sciences · doi:10.3390/educsci13050529 · Cited by 31
Exploring political discussions by Korean twitter users
Choi, M., Sang, Y., & Park, H. W. (2014) · Aslib Journal of Information Management · doi:10.1108/ajim-11-2012-0089 · Cited by 31
Activities in Retirement: Individual Experience of Silver Work
Maxin, L., & Deller, J. (2011) · Comparative Population Studies · doi:10.12765/cpos-2010-18 · Cited by 31
The Place of International Human Resource Management in International Business
Welch, D. E., & Björkman, I. (2014) · Management International Review · doi:10.1007/s11575-014-0226-3 · Cited by 34
Liang, X., Tsai, J.-Y., Mattis, K., Konieczna, M., & Dunwoody, S. (2014) · Journal of Broadcasting & Electronic Media · doi:10.1080/08838151.2014.906436 · Cited by 31
“She chose us to be your parents”
Hays, A., Horstman, H. K., Colaner, C. W., & Nelson, L. R. (2015) · Journal of Social and Personal Relationships · doi:10.1177/0265407515611494 · Cited by 31
de Ruijter, D., Smit, E. S., de Vries, H., Goossens, L., & Hoving, C. (2017) · BMJ Open · doi:10.1136/bmjopen-2016-014154 · Cited by 31
What makes a word difficult? Insights into the mental representation of technical terms
Jucks, R., & Paus, E. (2011) · Metacognition and Learning · doi:10.1007/s11409-011-9084-6 · Cited by 31
Indexing reliability for condition survey data
Taylor, J., & Watkinson, D. E. (2007) · Journal of the Institute of Conservation · doi:10.1080/01410096.2007.9995223 · Cited by 31
Meyer, F., Meissel, K., & McNaughton, S. (2017) · Journal of Research in Reading · doi:10.1111/1467-9817.12061 · Cited by 30
Levinthal, C., Kuusisto, E., & Tirri, K. (2021) · Education Sciences · doi:10.3390/educsci11060306 · Cited by 30
Cavazos‐Rehg, P. A., Fitzsimmons‐Craft, E. E., Krauss, M. J., Anako, N., Xu, C., Kasson, E., Costello, S. J., & Wilfley, D. E. (2020) · International Journal of Eating Disorders · doi:10.1002/eat.23282 · Cited by 30
Scamming higher ed: An analysis of phishing content and trends
Morrow, E. (2024) · Computers in Human Behavior · doi:10.1016/j.chb.2024.108274 · Cited by 30
Paus, E., Werner, C. S., & Jucks, R. (2011) · Computers & Education · doi:10.1016/j.compedu.2011.12.008 · Cited by 30
Private Prisons in Public Discourse: Measuring Moral Legitimacy
Burkhardt, B. C. (2014) · Sociological Focus · doi:10.1080/00380237.2014.940264 · Cited by 30
Disclosure on Measures to Prevent Corruption Risks: A Study of Italian Local Governments
D’Onza, G., Brotini, F., & Zarone, V. (2016) · International Journal of Public Administration · doi:10.1080/01900692.2016.1143000 · Cited by 30
A Data-Driven Approach to Evaluation of Sustainability Reporting Practices in Extractive Industries
Demirkan, C. P., Smith, N., Düzgün, Ş., & Waclawski, A. (2021) · Sustainability · doi:10.3390/su13168716 · Cited by 30
A Classroom-Based Study on the Antecedents of Epistemic Curiosity in L2 Learning
Nakamura, S., Reinders, H., & Darasawang, P. (2022) · Journal of Psycholinguistic Research · doi:10.1007/s10936-022-09839-x · Cited by 30
Szolnoki, G., & Tafel, M. (2022) · Sustainability · doi:10.3390/su141911831 · Cited by 30
Katz, P., Radvanski, D. C., Allen, D., Buyske, S. G., Schiff, S. A., Nadkarni, A., Rosenblatt, L. C., Maclean, R., & Hassett, A. L. (2011) · Arthritis Care & Research · doi:10.1002/acr.20617 · Cited by 30
Santiago-Brown, I., Metcalfe, A. V., Jerram, C., & Collins, C. (2014) · Sustainability · doi:10.3390/su6042031 · Cited by 30
Arousal and valence prediction in spontaneous emotional speech: felt versus perceived emotion
Truong, K. P., van Leeuwen, D. A., Neerincx, M. A., & de Jong, F. M. G. (2009) · Interspeech (USB) · doi:10.21437/interspeech.2009-583 · Cited by 30
Diabetes specialist nurses’ perceptions of their multifaceted role
Boström, E., Isaksson, U., Lundman, B. M., Sjölander, A. E., & Hörnsten, Å. (2012) · European Diabetes Nursing · doi:10.1002/edn.204 · Cited by 30
GRADHIST — A method for detection and analysis of oceanic fronts from remote sensing data
Kirches, G., Paperin, M., Klein, H. A., Brockmann, C., & Stelzer, K. (2016) · Remote Sensing of Environment · doi:10.1016/j.rse.2016.04.009 · Cited by 30
Content Analysis of Trends in Print Magazine Tobacco Advertisements
Banerjee, S. C., Shuk, E., Greene, K. L., & Ostroff, J. S. (2015) · Tobacco Regulatory Science · doi:10.18001/trs.1.2.1 · Cited by 30
College student misbehaviors: an exploration of instructor perceptions
Johnson, Z. D., Claus, C. J., Goldman, Z. W., & Sollitto, M. (2016) · Communication Education · doi:10.1080/03634523.2016.1202995 · Cited by 30
Cairns, A. W., Bond, R., Finlay, D., Breen, C. J., Guldenring, D., Gaffney, R., Gallagher, A. G., Peace, A. J., & Henn, P. (2016) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2016.09.016 · Cited by 30
Gawronski, A., Kuzmanovic, B., Georgescu, A. L., Kockler, H., Lehnhardt, F. G., Schilbach, L., Volpert, K., & Vogeley, K. T. (2011) · Fortschritte der Neurologie · Psychiatrie · doi:10.1055/s-0031-1281734 · Cited by 33
Scope of Self: Toward a Model of Television's Effects on Self-Complexity in Adolescence
Harrison, K. (2006) · Communication Theory · doi:10.1111/j.1468-2885.2006.00270.x · Cited by 30
Generalizability of Clinical Trial Results for Adolescent Major Depressive Disorder
Blanco, C., Hoertel, N., Franco, S., Olfson, M., He, J.-P., López, S., González-Pinto, A., Limosin, F., & Merikangas, K. R. (2017) · Pediatrics · doi:10.1542/peds.2016-1701 · Cited by 29
Populist discourse and active metaphors in the 2016 US presidential elections
Keating, J. (2021) · Intercultural Pragmatics · doi:10.1515/ip-2021-4004 · Cited by 29
Working around: Job crafting in the context of public and professional accountability
Renkema, E., Broekhuis, M., Tims, M., & Ahaus, K. (2022) · Human Relations · doi:10.1177/00187267221104011 · Cited by 29
Zhang, S., Zhang, Y., Li, J., Ni, Z., & Liu, Z. (2024) · Computers in Human Behavior · doi:10.1016/j.chb.2024.108136 · Cited by 31
Appraise: an Open-Source Toolkit for Manual Evaluation of MT Output
Federmann, C. (2012) · The Prague Bulletin of Mathematical Linguistics · doi:10.2478/v10108-012-0006-9 · Cited by 29
Content Analysis of Diet-Related Mobile Apps: A Self-Regulation Perspective
Zahry, N. R., Cheng, Y., & Peng, W. (2016) · Health Communication · doi:10.1080/10410236.2015.1072123 · Cited by 29
Conquering the Challenge of Continuous Business Model Improvement
Simmert, B., Ebel, P., Peters, C., Bittner, E. A. C., & Leimeister, J. M. (2018) · Business & Information Systems Engineering · doi:10.1007/s12599-018-0556-y · Cited by 29
The “Risser+” grade: a new grading system to classify skeletal maturity in idiopathic scoliosis
Troy, M. J., Miller, P. E., Price, N., Talwalkar, V. R., Zaina, F., Donzelli, S., Négrini, S., & Hresko, M. T. (2018) · European Spine Journal · doi:10.1007/s00586-018-5821-8 · Cited by 29
Beckler, D. T., Thumser, Z. C., Schofield, J. S., & Marasco, P. D. (2018) · BMC Medical Research Methodology · doi:10.1186/s12874-018-0606-7 · Cited by 31
Increasingly sexy? Sexuality and sexual objectification in popular music videos, 1995–2016.
Karsay, K., Matthes, J., Buchsteiner, L., & Grosser, V. (2018) · Psychology of Popular Media Culture · doi:10.1037/ppm0000221 · Cited by 29
Piñeiro-Naval, V., & Mangana, R. (2019) · Palabra Clave · doi:10.5294/pacla.2019.22.1.6 · Cited by 29
Mapping the Emotional Experience of Travel to Understand Cycle-Transit User Behavior
Meenar, M., Flamm, B., & Keenan, K. (2019) · Sustainability · doi:10.3390/su11174743 · Cited by 29
A Method to Analyze Multiple Social Identities in Twitter Bios
Pathak, A., Madani, N., & Joseph, K. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3479502 · Cited by 32
Esau, K., & Frieß, D. (2021) · Journal of Deliberative Democracy · doi:10.16997/jdd.1021 · Cited by 32
Gender bias in high stakes pitching: an NLP approach
Khurana, I., & Lee, D. J. (2022) · Small Business Economics · doi:10.1007/s11187-021-00598-y · Cited by 32
Guidarelli, C., Lipps, C., Stoyles, S. A., Dieckmann, N. F., & Winters‐Stone, K. M. (2022) · Journal of Geriatric Oncology · doi:10.1016/j.jgo.2022.02.002 · Cited by 29
Draws, T., Inel, O., Tintarev, N., Baden, C., & Timmermans, B. (2022) · Conference on Human Information Interaction and Retrieval · doi:10.1145/3498366.3505812 · Cited by 35
Commentary-A Dissenting View on So-Called Paradoxes of Reliability Coefficients
Salmon, C. T. (2012) · Routledge eBooks · doi:10.4324/9780203113653-24 · Cited by 29
What do users care about? Research on user behavior of mobile interactive video advertising
Gu, C., Lin, S., Sun, J. L., Yang, C. K., Chen, J., Jiang, Q., Miao, W., & Wei, W. (2022) · Heliyon · doi:10.1016/j.heliyon.2022.e10910 · Cited by 29
On the perceived usefulness of risk descriptions for decision-making in disaster risk management
Lin, L., Nilsson, A., Sjölin, J., Abrahamsson, M., & Tehler, H. (2015) · Reliability Engineering & System Safety · doi:10.1016/j.ress.2015.04.012 · Cited by 29
TELPS: A method for analysing mathematics pre-service teachers' Pedagogical Content Knowledge
Prescott, A. E., Bausch, I., & Bruder, R. (2013) · Teaching and Teacher Education · doi:10.1016/j.tate.2013.05.002 · Cited by 31
Tapaninen, A., Seppanen, M., & Mäkinen, S. (2009) · International Journal of Agile Systems and Management · doi:10.1504/ijasm.2009.023250 · Cited by 29
Political agency of news outlets in a polarized media system: Framing the corruption probe in Turkey
Panayırcı, U. C., İşeri, E., & Şekercioğlu, E. (2016) · European Journal of Communication · doi:10.1177/0267323116669455 · Cited by 29
Nguyen‐Kim, T. D. L., Maurer, B., Suliman, Y. A., Morsbach, F., Distler, O., & Frauenfelder, T. (2018) · Journal of Thoracic Disease · doi:10.21037/jtd.2018.04.39 · Cited by 29
Why Do Students Misbehave? An Initial Examination of Antecedents to Student Misbehavior
Johnson, Z. D., Goldman, Z. W., & Claus, C. J. (2018) · Communication Quarterly · doi:10.1080/01463373.2018.1483958 · Cited by 29
Madkins, K., Moskowitz, D., Moran, K., Dellucci, T. V., & Mustanski, B. S. (2019) · AIDS Education and Prevention · doi:10.1521/aeap.2019.31.4.287 · Cited by 29
Racially liberatory pedagogy: a Black Lives Matter approach to education
Castillo‐Montoya, M., Abreu, J., & Abad, A. (2019) · International Journal of Qualitative Studies in Education · doi:10.1080/09518398.2019.1645904 · Cited by 31
The psychological mechanisms of persuasive impact from narrative communication
Shen, L., Seung, S., Andersen, K. K., & McNeal, D. M. (2018) · Studies in Communication Sciences · doi:10.24434/j.scoms.2017.02.003 · Cited by 29
Is critical thinking happening? Testing content analysis schemes applied to MOOC discussion forums
O’Riordan, T., Millard, D. E., & Schulz, J. (2020) · Computer Applications in Engineering Education · doi:10.1002/cae.22314 · Cited by 32
The YouTube Indian: Portrayals of Native Americans on a viral video site
Kopacz, M., & Lawton, B. L. (2010) · New Media & Society · doi:10.1177/1461444810373532 · Cited by 29
A Categorical Annotation Scheme for Emotion in the Linguistic Content of Dialogue
Craggs, R., & Wood, M. M. (2004) · Lecture notes in computer science · doi:10.1007/978-3-540-24842-2_9 · Cited by 29
Annotations and subjective machines
Reidsma, D. (2008) · doi:10.3990/1.9789036527262 · Cited by 29
Möri, M., & Fahr, A. (2023) · Frontiers in Psychology · doi:10.3389/fpsyg.2023.1297687 · Cited by 28
Teachers’ diagnosis of students’ research skills during the mentoring of the undergraduate thesis
Agricola, B. T., Prins, F. J., van der Schaaf, M. F., & van Tartwijk, J. (2018) · Mentoring & Tutoring Partnership in Learning · doi:10.1080/13611267.2018.1561015 · Cited by 28
Davis, L. F., Ramírez‐Andreotta, M. D., & Buxner, S. R. (2020) · Citizen Science Theory and Practice · doi:10.5334/cstp.253 · Cited by 28
Wilson, C., Claussen, C., & Valverde, P. (2021) · Resources Policy · doi:10.1016/j.resourpol.2021.102084 · Cited by 28
Franzoni, C., & Tenca, F. (2022) · Entrepreneurship Theory and Practice · doi:10.1177/10422587221102107 · Cited by 29
Cappelli, L., Pisano, A., Iannucci, E., Papetti, P., D’Ascenzo, F., & Ruggieri, R. (2023) · Business Strategy and the Environment · doi:10.1002/bse.3332 · Cited by 30
Cash, P. J., Gonçalves, M., & Dorst, K. (2023) · Design Studies · doi:10.1016/j.destud.2023.101219 · Cited by 28
The Changing World of Mobile Communications
Ahokangas, P., & Aagaard, A. (2023) · Springer eBooks · doi:10.1007/978-3-031-33191-6 · Cited by 28
Metadata
- Authors
- Krippendorff, K.
- Corresponding author
- Krippendorff, K.
- ORCID
- Krippendorff, K.
- Venue
- Human Communication Research
- Publisher
- Oxford University Press (OUP)
- Published
- 2004-07-01
- Volume
- 30
- Issue
- 3
- Pages
- 411-433
- ISSN
- 0360-3989 (ISSN-L)
- 1468-2958
- OpenAlex
- W4243989635
- Cited by
- 1,027
- References
- 0
- Citations by year
- 2026: 42 · 2025: 100 · 2024: 108 · 2023: 108 · 2022: 124 · 2021: 135 · 2020: 105 · 2019: 57 · 2018: 44 · 2017: 29 · 2016: 41 · 2015: 25 · 2014: 26 · 2013: 21 · 2012: 15
- FWCI
- 5.93
- Type
- journal article
- Language
- en
- Open access
- green
- repository.upenn.edu
- Locations
- Human Communication Research (publishedVersion)
- ScholarlyCommons (University of Pennsylvania) (submittedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Reliability and Agreement in Measurement
- Social Sciences › General Social Sciences › Computational and Text Analysis Methods
- Social Sciences › Decision Sciences › Management Science and Operations Research › Psychometric Methodologies and Testing
- Keywords
- inter-rater reliability, reliability coefficients, interobserver agreement, reporting reliability
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
