Cited by 3,068Reliability and Agreement in MeasurementNews Media Framing and Political CommunicationComputational and Text Analysis Methods

The DOI matches its Crossref record; 42 of the 42 references it cites were found in a registry or an index.

Content Analysis in Mass Communication: Assessment and Reporting of Intercoder Reliability

Lombard, Matthew, Snyder-Duch, Jennifer, Bracken, Cheryl Campanella·2002·Human Communication Research

DOI 10.1111/j.1468-2958.2002.tb00826.x

Open this paper in CoChat — free account

Abstract

As a method specifically intended for the study of messages, content analysis is fundamental to mass communication research. Intercoder reliability, more specifically termed intercoder agreement, is a measure of the extent to which independent judges make the same coding decisions in evaluating the characteristics of messages, and is at the heart of this method. Yet there are few standard and accessible guidelines available regarding the appropriate procedures to use to assess and report intercoder reliability, or software tools to calculate it. As a result, it seems likely that there is little consistency in how this critical element of content analysis is assessed and reported in published mass communication studies. Following a review of relevant concepts, indices, and tools, a content analysis of 200 studies utilizing content analysis published in the communication literature between 1994 and 1998 is used to characterize practices in the field. The results demonstrate that mass communication researchers often fail to assess (or at least report) intercoder reliability and often rely on percent agreement, an overly liberal index. Based on the review and these results, concrete guidelines are offered regarding procedures for assessment and reporting of this important aspect of content analysis.

Integrity signals

Registry

Crossref · 10.1111/j.1468-2958.2002.tb00826.x

Cites

42 of 42 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

A Coefficient of Agreement for Nominal Scales
Cohen, J. (1960) · Educational and Psychological Measurement · doi:10.1177/001316446002000104 · Cited by 42,398
Content Analysis: An Introduction to its Methodology.
Shelley, M., & Krippendorff, K. (1984) · Journal of the American Statistical Association · doi:10.2307/2288384 · Cited by 24,351
The Content Analysis Guidebook
Neuendorf, K. A. (2017) · doi:10.4135/9781071802878 · Cited by 9,286 · cited 10×
“), but this technique has also been questioned (Neuendorf, 2002). With the coding data in hand, the researcher calculates and reports one or more indices of reliability. Popping (1988) identified 39 different “agreement indices” for coding nominal categories, which excludes several techniques for ratio and interval le…”
BackgroundMethodInfluential
A concordance correlation coefficient to evaluate reproducibility.
Lin, L. I.-K. (1989) · Biometrics · doi:10.2307/2532051 · Cited by 8,785
Weighted kappa: nominal scale agreement provision for scaled disagreement or partial credit
Cohen, J. (1968) · Psychological Bulletin · doi:10.1037/h0026256 · Cited by 8,736
Measuring nominal scale agreement among many raters.
Fleiss, J. L. (1971) · Psychological Bulletin · doi:10.1037/h0031619 · Cited by 8,733
Content Analysis for the Social Sciences and Humanities.
Parker, E. B., & Holsti, O. R. (1970) · American Sociological Review · doi:10.2307/2093233 · Cited by 4,553
Content Analysis in Communication Research.
Pool, I. D. S., & Berelson, B. (1952) · American Sociological Review · doi:10.2307/2088029 · Cited by 3,106
Content Analysis for the Social Sciences and Humanities
Parker, S. R. (1970) · Sociology · doi:10.1177/003803857000400231 · Cited by 2,178
Handbook of Research Methods in Social and Personality Psychology
West, T., Reis, H. T., Judd, C. M., Banaji, M. R., Crandall, C., Giner-Sorolla, R., Biernat, M., Fiedler, K., Ermark, F., Tepper, S. J., et al. (2024) · Cambridge University Press eBooks · doi:10.1017/9781009170123 · Cited by 2,134

Show all 42 references

Research works printed but not linked: 2

Lacy S., 1996, Journalism & Mass Communication Quarterly, 73, 969
Perreault W. D., 1989, Journal of Marketing Research, 26, 135, 10.1177/002224378902600201 · DOI · Paper page

Sorted by citations

Items 301–400 of 1,005

Exploring viewer participation in online video game streaming: A mixed-methods approach
Xu, X., Luo, X. R., Wu, K., & Zhao, W. (2021) · International Journal of Information Management · doi:10.1016/j.ijinfomgt.2020.102297 · Cited by 82
Ambidextrous leadership: a meta-review applying static and dynamic multi-level perspectives
Mueller, J., Renzl, B. E., & Will, M. G. (2018) · Review of Managerial Science · doi:10.1007/s11846-018-0297-9 · Cited by 83
Content and Effects of News Stories About Uncertain Cancer Causes and Preventive Behaviors
Niederdeppe, J., Lee, T. T. H., Robbins, R., Kim, H. K., Kresovich, A., Kirshenblat, D., Standridge, K., Clarke, C. E., Jensen, J. D., & Fowler, E. F. (2013) · Health Communication · doi:10.1080/10410236.2012.755603 · Cited by 74
Cyber warriors in the Middle East: The case of the Syrian Electronic Army
Al‐Rawi, A. (2014) · Public Relations Review · doi:10.1016/j.pubrev.2014.04.005 · Cited by 79
Interactive Behavior in Political Discussions on Twitter: Politicians, Media, and Citizens’ Patterns of Interaction in the 2015 and 2016 Electoral Campaigns in Spain
Guerrero-Solé, F. (2018) · Social Media + Society · doi:10.1177/2056305118808776 · Cited by 75
A Polyhedron Model of Wisdom: A Systematic Review of the Wisdom Studies in Psychology, Management and Leadership, and Education
Karami, S., Ghahremani, M., Parra-Martínez, F. A., & Gentry, M. (2020) · Roeper Review · doi:10.1080/02783193.2020.1815263 · Cited by 81
Drivers of consumer purchase intentions for remanufactured products
Gaur, J., Amini, M., Banerjee, P., & Gupta, R. (2015) · Qualitative Market Research An International Journal · doi:10.1108/qmr-01-2014-0001 · Cited by 79
Merging content analysis and the policy sciences: A system to discern policy-specific trends from news media reports
Howland, D., Becker, M. L., & Prelli, L. J. (2006) · Policy Sciences · doi:10.1007/s11077-006-9016-5 · Cited by 73
Attitudes towards the role of Cost–Benefit Analysis in the decision-making process for spatial-infrastructure projects: A Dutch case study
Mouter, N., Annema, J. A., & van Wee, B. (2013) · Transportation Research Part A Policy and Practice · doi:10.1016/j.tra.2013.10.006 · Cited by 73
Targeting the farmer decision making process: A pathway to increased adoption of integrated weed management
Wilson, R. S., Hooker, N. H., Tucker, M., LeJeune, J., & Doohan, D. (2009) · Crop Protection · doi:10.1016/j.cropro.2009.05.013 · Cited by 73
A Friendships and Dating Program for Adults with Intellectual and Developmental Disabilities: A Formative Evaluation
Ward, K. M., Atkinson, J. P., Smith, C. A., & Windsor, R. A. (2013) · Intellectual and developmental disabilities · doi:10.1352/1934-9556-51.01.022 · Cited by 74
What Does the Media Mean by ‘Sustainability’ or ‘Sustainable Development’? an Empirical Analysis of Sustainability Terminology in German Newspapers Over Two Decades
Fischer, D., Haucke, F. V., & Sundermann, A. (2017) · Sustainable Development · doi:10.1002/sd.1681 · Cited by 73
User-generated place brand identity: harnessing the power of content on social media platforms
Taecharungroj, V. (2018) · Journal of Place Management and Development · doi:10.1108/jpmd-11-2017-0117 · Cited by 72
Using virtual standardized patients to accurately assess information gathering skills in medical students
Maicher, K. R., Zimmerman, L., Wilcox, B., Liston, B., Cronau, H., Macerollo, A., Jin, L., Jaffe, E., White, M., Fosler-Lussier, E., et al. (2019) · Medical Teacher · doi:10.1080/0142159x.2019.1616683 · Cited by 72
Collectivist and Individualist Influences on Website Design in South Korea and the U.S.: A Cross-Cultural Content Analysis
Kim, H., Coyle, J. R., & Gould, S. J. (2009) · Journal of Computer-Mediated Communication · doi:10.1111/j.1083-6101.2009.01454.x · Cited by 72
Pin It
Hambrick, M. E., & Kang, S. (2014) · Communication & Sport · doi:10.1177/2167479513518044 · Cited by 72
Food crisis coverage by social and traditional media: A case study of the 2008 Irish dioxin crisis
Shan, L. C., Regan, Á., Brún, A. D., Barnett, J. C., van der Sanden, M. C. A., Wall, P. G., & McConnon, Á. (2013) · Public Understanding of Science · doi:10.1177/0963662512472315 · Cited by 72
University life with ASD: Faculty knowledge and student needs
Zeedyk, S. M., Bolourian, Y., & Blacher, J. (2018) · Autism · doi:10.1177/1362361318774148 · Cited by 81
The Ball Game Is for the Boys: The Visual Framing of Female Athletes on National Sports Networks’ Instagram Accounts
Romney, M., & Johnson, R. G. (2019) · Communication & Sport · doi:10.1177/2167479519836731 · Cited by 71
Delegitimizing the media?
Egelhofer, J. L., Aaldering, L., & Lecheler, S. (2021) · Journal of Language and Politics · doi:10.1075/jlp.20081.ege · Cited by 56
The Importance of Corporate Reputation for Sustainable Supply Chains: A Systematic Literature Review, Bibliometric Mapping, and Research Agenda
von Berlepsch, D., Lemke, F., & Gorton, M. (2022) · Journal of Business Ethics · doi:10.1007/s10551-022-05268-x · Cited by 98
What Drives User Stickiness and Satisfaction in OTT Video Streaming Platforms? A Mixed-Method Exploration
Periaiya, S., & Nandukrishna, A. T. (2023) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2022.2160224 · Cited by 95
Pandering, protesting, engaging. Norwegian party leaders on Facebook during the 2013 ‘Short campaign'
Larsson, A. O. (2014) · Information Communication & Society · doi:10.1080/1369118x.2014.967269 · Cited by 71
AGENCIFICATION AND BLAME SHIFTING: EVALUATING A NEGLECTED SIDE OF PUBLIC SECTOR REFORMS
Mortensen, P. B. (2016) · Public Administration · doi:10.1111/padm.12243 · Cited by 71
Defining sport tourism: a content analysis of an evolving epistemology
Rheenen, D. V., Cernăianu, S., & Sobry, C. (2016) · Journal of Sport & Tourism · doi:10.1080/14775085.2016.1229212 · Cited by 71
“Wow! It's cool”: the meaning of coolness in marketing
Rahman, K. (2013) · Marketing Intelligence & Planning · doi:10.1108/mip-09-2012-0094 · Cited by 70
Emerging trends in data analytics and knowledge management job market: extending KSA framework
Chang, H., Wang, C., & Hawamdeh, S. (2018) · Journal of Knowledge Management · doi:10.1108/jkm-02-2018-0088 · Cited by 70
Post-failure impression management: A typology of entrepreneurs’ public narratives after business closure
Kibler, E., Mandl, C., Farny, S., & Salmivaara, V. (2020) · Human Relations · doi:10.1177/0018726719899465 · Cited by 78
Nudity of female and male models in primetime TV advertising across seven countries
Nelson, M. R., & Paek, H. (2008) · International Journal of Advertising · doi:10.2501/s0265048708080281 · Cited by 67
The effect of abductive reasoning on concept selection decisions
Dong, A., Lovallo, D. P., & Mounarath, R. (2015) · Design Studies · doi:10.1016/j.destud.2014.12.004 · Cited by 70
Hashtagging Politics: Transnational Anti-Fracking Movement Twitter Practices
Hopke, J. E. (2015) · Social Media + Society · doi:10.1177/2056305115605521 · Cited by 69
The textbook task as a genre
Bakken, J., & Andersson-Bakken, E. (2021) · Journal of Curriculum Studies · doi:10.1080/00220272.2021.1929499 · Cited by 94
Authentic STEM research, practices of science, and interest development in an informal science education program
Habig, B., & Gupta, P. (2021) · International Journal of STEM Education · doi:10.1186/s40594-021-00314-y · Cited by 77
A systematic review of technologies and standards used in the development of rule-based clinical decision support systems
Papadopoulos, P., Soflano, M., Chaudy, Y., Adejo, W., & Connolly, T. M. (2022) · Health and Technology · doi:10.1007/s12553-022-00672-9 · Cited by 95
Designs for explaining intelligent agents
Haynes, S. R., Cohen, M. A., & Ritter, F. (2009) · Int. J. Hum. Comput. Stud. · doi:10.1016/j.ijhcs.2008.09.008 · Cited by 75
Coalition Stability and Belief Change: Advocacy Coalitions in U.S. Foreign Policy and the Creation of Israel, 1922–44
Pierce, J. J. (2011) · Policy Studies Journal · doi:10.1111/j.1541-0072.2011.00415.x · Cited by 67
Three Decades of Reliability in Communication Content Analyses
Lovejoy, J., Watson, B. R., Lacy, S. R., & Riffe, D. (2016) · Journalism & Mass Communication Quarterly · doi:10.1177/1077699016644558 · Cited by 97
Thematic trends in event management research
Park, S. B., & Park, K. (2017) · International Journal of Contemporary Hospitality Management · doi:10.1108/ijchm-09-2015-0521 · Cited by 68
Identification of drivers, benefits, and challenges of ISO 50001 through case study content analysis
Fuchs, H., Aghajanzadeh, A., & Therkelsen, P. L. (2020) · Energy Policy · doi:10.1016/j.enpol.2020.111443 · Cited by 67
Effect of message orientation/vividness on consumer engagement for travel brands on social networking sites
Yousaf, A., Amin, I., Jaziri, D., & Mishra, A. (2020) · Journal of Product & Brand Management · doi:10.1108/jpbm-08-2019-2546 · Cited by 79
Non-financial key performance indicators: what determines the differences in the quality and quantity of the disclosures?
Zarzycka, E., & Krasodomska, J. (2021) · Journal of Applied Accounting Research · doi:10.1108/jaar-02-2021-0036 · Cited by 67
Coverage of the Iraq War in the United States, Mainland China, Taiwan and Poland
Гуо, Л., Chen, Y.-N. K., Vu, H. T., Wang, Q., Aksamit, R., Guzek, D., Jachimowski, M., & McCombs, M. (2014) · Journalism Studies · doi:10.1080/1461670x.2014.885198 · Cited by 67
A Call for Standardization in Content Analysis Reliability
Lombard, M. (2004) · Human Communication Research · doi:10.1111/j.1468-2958.2004.tb00739.x · Cited by 67
The Mass Media and the Public's Assessments of Presidential Candidates, 1952–2000
Gilens, M., Vavreck, L., & Cohen, M. (2007) · The Journal of Politics · doi:10.1111/j.1468-2508.2007.00615.x · Cited by 71
Climate Change Counter Movement Neutralization Techniques: A Typology to Examine the Climate Change Counter Movement
McKie, R. E. (2018) · Sociological Inquiry · doi:10.1111/soin.12246 · Cited by 67
Social Media Content Analysis of Public Diabetes Facebook Groups
Stellefson, M., Paige, S. R., Apperson, A., & Spratt, S. (2019) · Journal of Diabetes Science and Technology · doi:10.1177/1932296819839099 · Cited by 68
Professional Development Through Engineering Academies: An Examination of Elementary Teachers' Recognition and Understanding of Engineering
Duncan, D., Diefes‐Dux, H. A., & Gentry, M. (2011) · Journal of Engineering Education · doi:10.1002/j.2168-9830.2011.tb00025.x · Cited by 66
The Challenges of Remote K–12 Education During the COVID-19 Pandemic: Differences by Grade Level
Leech, N. L., Gullett, S., Cummings, M. H., & Haug, C. A. (2022) · Online Learning · doi:10.24059/olj.v26i1.2609 · Cited by 66
Interrater reliability estimators tested against true interrater reliabilities
Zhao, X., Feng, G. C., Ao, S. H., & Liu, P. L. (2022) · BMC Medical Research Methodology · doi:10.1186/s12874-022-01707-5 · Cited by 93
What is on your mind? Using the perceptual cycle model and critical decision method to understand the decision-making process in the cockpit
Plant, K. L., & Stanton, N. A. (2013) · Ergonomics · doi:10.1080/00140139.2013.809480 · Cited by 66
Does Avatar Appearance Matter? How Team Visual Similarity and Member-Avatar Similarity Influence Virtual Team Performance
van der Land, S. F., Schouten, A. P., Feldberg, F., Huysman, M., & van den Hooff, B. (2014) · Human Communication Research · doi:10.1111/hcre.12044 · Cited by 72
How do libraries use social networking sites to interact with users
Chen, D. Y., Chu, S. K. W., & Xu, S. (2012) · Proceedings of the American Society for Information Science and Technology · doi:10.1002/meet.14504901085 · Cited by 66
Victim Ambiguity: Bystander Intervention and Sexual Assault in the College Drinking Scene
Pugh, B., Ningard, H., Ven, T. V., & Butler, L. C. (2016) · Deviant Behavior · doi:10.1080/01639625.2015.1026777 · Cited by 66
Parent and Provider Perspectives on Procedural Care for Children with Autism Spectrum Disorders
Davignon, M. N., Friedlaender, E., Cronholm, P. F., Paciotti, B., & Levy, S. E. (2014) · Journal of Developmental & Behavioral Pediatrics · doi:10.1097/dbp.0000000000000036 · Cited by 66
Students’ development of socio-scientific reasoning in a mobile augmented reality learning environment
Chang, H., Hsu, Y., Wu, H., & Tsai, C. (2018) · International Journal of Science Education · doi:10.1080/09500693.2018.1480075 · Cited by 72
Obesity frames and counter-frames in British and German online newspapers
Atanasova, D., & Koteyko, N. (2016) · Health An Interdisciplinary Journal for the Social Study of Health Illness and Medicine · doi:10.1177/1363459316649764 · Cited by 65
Evaluating large language models in analysing classroom dialogue
Long, Y., Luo, H., & Zhang, Y. (2024) · npj Science of Learning · doi:10.1038/s41539-024-00273-3 · Cited by 87
Some views on agreement to be used in content analysis studies
Popping, R. (2009) · Quality & Quantity · doi:10.1007/s11135-009-9258-3 · Cited by 65
A content analysis of a “boy love” support forum: Revisiting Durkin and Bryant
O’Halloran, E., & Quayle, E. (2010) · Journal of Sexual Aggression · doi:10.1080/13552600903395319 · Cited by 65
Strategic Planners in More Turbulent Times: The Changing Job Characteristics of Strategy Professionals, 1960–2003
Whittington, R., Yakis‐Douglas, B., Ahn, K., & Cailluet, L. (2016) · Long Range Planning · doi:10.1016/j.lrp.2015.12.021 · Cited by 65
Evolution of newspaper coverage of water issues in Australia during 1843–2011
Wei, J., Wei, Y., Western, A. W., Skinner, D., & Lyle, C. (2014) · AMBIO · doi:10.1007/s13280-014-0571-2 · Cited by 64
An analysis of the global additive manufacturing response to the COVID-19 pandemic
Kunovjanek, M., & Wankmüller, C. (2020) · Journal of Manufacturing Technology Management · doi:10.1108/jmtm-07-2020-0263 · Cited by 66
Women, Weight, and Age: Social Comparison to Magazine Images Across the Lifespan
Bessenoff, G. R., & Priore, R. E. D. (2007) · Sex Roles · doi:10.1007/s11199-006-9164-2 · Cited by 64
Navigating the unknown: A content analysis of the unique challenges faced by adults with rare diseases
Bryson, B., Bogart, K., Atwood, M., Fraser, K., Locke, T., Pugh, K., & Zerrouk, M. (2021) · Journal of Health Psychology · doi:10.1177/1359105319828150 · Cited by 63
Assumptions behind Intercoder Reliability Indices
Zhao, X., Liu, J. S., & Deng, K. (2013) · Annals of the International Communication Association · doi:10.1080/23808985.2013.11679142 · Cited by 113
Necessary But Not Sufficient
Lund, E. M., & Thomas, K. B. (2015) · Psychology of Women Quarterly · doi:10.1177/0361684315598286 · Cited by 63
Would You Be My Friend? An Examination of Global Marketers' Brand Personification Strategies in Social Media
Chen, K., Lin, J.-S., Choi, J. H., & Hahm, J. M. (2015) · Journal of Interactive Advertising · doi:10.1080/15252019.2015.1079508 · Cited by 72
Auditing Information Structures in Organizations: A Review of Data Collection Techniques for Network Analysis
Zwijze‐Koning, K. H., & de Jong, M. D. T. (2005) · Organizational Research Methods · doi:10.1177/1094428105280120 · Cited by 62
On voluntarism and the role of governments in CSR : towards a contingency approach
Dentchev, N. A., van Balen, M., & Haezendonck, E. L. (2014) · Business Ethics A European Review · doi:10.1111/beer.12088 · Cited by 63
A Pictorial Analysis of Destination Images on Pinterest: The Case of Tokyo, Kyoto, and Osaka, Japan
Song, S.-G., & Kim, D.-Y. (2016) · Journal of Travel & Tourism Marketing · doi:10.1080/10548408.2016.1167384 · Cited by 64
Who Uses Direct-to-Consumer Brain Stimulation Products, and Why? A Study of Home Users of tDCS Devices
Wexler, A. (2017) · Journal of Cognitive Enhancement · doi:10.1007/s41465-017-0062-z · Cited by 63
Are the pens working for justice? News media coverage of renewable energy involving Indigenous Peoples in Canada
Walker, C., Alexander, A., Doucette, M. B., Lewis, D., NEUFELD, H. T., Martin, D. H., Masuda, J., Stefanelli, R. D., & Castleden, H. E. (2019) · Energy Research & Social Science · doi:10.1016/j.erss.2019.101230 · Cited by 64
Is Pornography Consumption a Risk Factor for Condomless Sex?
Tokunaga, R. S., Wright, P. J., & Vangeel, L. (2020) · Human Communication Research · doi:10.1093/hcr/hqaa005 · Cited by 69
How We Code
Shaffer, D. W., & Ruis, A. R. (2021) · Communications in computer and information science · doi:10.1007/978-3-030-67788-6_5 · Cited by 62
Development and evaluation of social indicators of vulnerability and resiliency for fishing communities in the Gulf of Mexico
Jacob, S., Weeks, P., Blount, B. G., & Jepson, M. (2012) · Marine Policy · doi:10.1016/j.marpol.2012.04.014 · Cited by 62
StarvingforPerfect.com: A Theoretically Based Content Analysis of Pro-Eating Disorder Web Sites
Lapinski, M. K. (2006) · Health Communication · doi:10.1207/s15327027hc2003_4 · Cited by 62
Knowledge and processes that predict proficiency in digital literacy
Bulger, M., Mayer, R. E., & Metzger, M. J. (2014) · Reading and Writing · doi:10.1007/s11145-014-9507-2 · Cited by 78
More of the Same or Marketplace of Opinions? A Cross-National Comparison of Diversity in Online News Reporting
Humprecht, E., & Büchel, F. (2013) · The International Journal of Press/Politics · doi:10.1177/1940161213497595 · Cited by 62
Anti‐consumption as a means to save jobs
Hoffmann, S. (2011) · European Journal of Marketing · doi:10.1108/03090561111167342 · Cited by 62
Newspaper coverage of water issues in Australia
Hürlimann, A., & Dolničar, S. (2012) · Water Research · doi:10.1016/j.watres.2012.09.028 · Cited by 62
iObjectify: Self‐ and other‐objectification on Grindr, a geosocial networking application designed for men who have sex with men
Anderson, J., Holland, E., Koç, Y., & Haslam, N. (2017) · European Journal of Social Psychology · doi:10.1002/ejsp.2350 · Cited by 62
The development of new environmental policies and processes in response to a crisis: the case of the multiple barrier approach for safe drinking water
Plummer, R., Velaniškis, J., de Grosbois, D., Kreutzwiser, R. D., & de Loë, R. (2010) · Environmental Science & Policy · doi:10.1016/j.envsci.2010.05.004 · Cited by 57
Disclosure on climate change: Analysing the UK ETS effects
de Aguiar, T. R. S., & Bebbington, J. (2014) · Accounting Forum · doi:10.1016/j.accfor.2014.10.002 · Cited by 61
Movie exposure to smoking cues and adolescent smoking onset: A test for mediation through peer affiliations.
Wills, T. A., Sargent, J. D., Stoolmiller, M., Gibbons, F. X., Worth, K. A., & Cin, S. D. (2007) · Health Psychology · doi:10.1037/0278-6133.26.6.769 · Cited by 61
A Review of Persuasive Principles in Mobile Apps for Chronic Arthritis Patients: Opportunities for Improvement
Geuens, J., Swinnen, T. W., Westhovens, R., de Vlam, K., Geurts, L., & Abeele, V. V. (2016) · JMIR mhealth and uhealth · doi:10.2196/mhealth.6286 · Cited by 62
The political economy of economic reform in the Pacific
Duncan, R. (2010) · ANU Open Research (Australian National University) · Cited by 59
Do messages spread widely also diffuse fast? Examining the effects of message characteristics on information diffusion
Zhu, X., Kim, Y., & Park, H. (2020) · Computers in Human Behavior · doi:10.1016/j.chb.2019.09.006 · Cited by 61
The concept of “presence” as a measure of ecological validity in driving simulators
Deniaud, C., Honnêt, V., Jeanne, B., & Mestre, D. (2015) · Journal of Interaction Science · doi:10.1186/s40166-015-0005-z · Cited by 60
Managerial perspectives on crowdsourcing in the new product development process
Zahay, D. L., Hajli, N., & Sihi, D. (2017) · Industrial Marketing Management · doi:10.1016/j.indmarman.2017.11.002 · Cited by 60
Cross-System Validation of Engagement Prediction from Log Files
Cocea, M., & Weibelzahl, S. (2007) · Lecture notes in computer science · doi:10.1007/978-3-540-75195-3_2 · Cited by 60
A systematic review of echo chamber research: comparative analysis of conceptualizations, operationalizations, and varying outcomes
Hartmann, D., Wang, S. M., Pohlmann, L., & Berendt, B. (2025) · Journal of Computational Social Science · doi:10.1007/s42001-025-00381-z · Cited by 59
Publicized Intimacies on Reality Television: An Analysis of Voyeuristic Content and Its Contribution to the Appeal of Reality Programming
Baruh, L. (2009) · Journal of Broadcasting & Electronic Media · doi:10.1080/08838150902907678 · Cited by 57
Situating evaluation in scenarios of use
Haynes, S. R., Purao, S., & Skattebo, A. L. (2004) · ACM Conference on Computer Supported Cooperative Work (CSCW) · doi:10.1145/1031607.1031624 · Cited by 59
How to activate teachers through teacher evaluation?
Tuytens, M., & Devos, G. (2013) · School Effectiveness and School Improvement · doi:10.1080/09243453.2013.842601 · Cited by 63
Exploring quality challenges and the validity of excellence models
Eriksson, H., Gremyr, I., Bergquist, B., Garvare, R., Fundin, A., Wiklund, H., Wester, M., & Sörqvist, L. (2016) · International Journal of Operations & Production Management · doi:10.1108/ijopm-12-2014-0610 · Cited by 59
“Let the community decide”? The vision and reality of soundness-only peer review in open-access mega-journals
Spezi, V., Wakeling, S., Pinfield, S., Fry, J., Creaser, C., & Willett, P. (2017) · Journal of Documentation · doi:10.1108/jd-06-2017-0092 · Cited by 64
Morphology of intervertebral disc ruptures evaluated by vacuum phenomenon using multi-detector computed tomography: association with lumbar disc degeneration and canal stenosis
Murata, K., Akeda, K., Takegami, N., Cheng, K. K., Masuda, K., & Sudo, A. (2018) · BMC Musculoskeletal Disorders · doi:10.1186/s12891-018-2086-7 · Cited by 59
Media coverage of management of the black bearUrsus thibetanusin Japan
Sakurai, R., Jacobson, S. K., & Carlton, J. S. (2013) · Oryx · doi:10.1017/s0030605312000890 · Cited by 58
A double-edged sword: Teachers’ perceptions of the benefits and challenges of online teaching and learning in higher education
Lucas, M., & Vicente, P. N. (2022) · Education and Information Technologies · doi:10.1007/s10639-022-11363-3 · Cited by 89
Service failures in online double deviation scenarios: justice theory approach
Lee, E. J., & Park, J. (2010) · Managing Service Quality · doi:10.1108/09604521011011621 · Cited by 61

Authors: Lombard, M., Snyder-Duch, J., & Bracken, C. C.
Affiliations: Temple University (US)
Cleveland State University (US)
Venue: Human Communication Research
Publisher: Oxford University Press (OUP)
Published: 2002-10
Volume: 28
Issue: 4
Pages: 587-604
ISSN: 0360-3989 (ISSN-L)
1468-2958
DOI: 10.1111/j.1468-2958.2002.tb00826.x
OpenAlex: W2160375265
Cited by: 3,068
References: 42
Citations by year: 2026: 234 · 2025: 173 · 2024: 136 · 2023: 143 · 2022: 212 · 2021: 238 · 2020: 206 · 2019: 206 · 2018: 174 · 2017: 168 · 2016: 174 · 2015: 153 · 2014: 168 · 2013: 158 · 2012: 117
FWCI: 19.75
Type: journal article
Language: en
Open access: closed
Licence: doi.wiley.com/10.1002/tdm_license_1.1 (tdm, 4718 d)
Locations: Human Communication Research (publishedVersion)
EngagedScholarship @ Cleveland State University (Cleveland State University) (submittedVersion)
Topics: Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Reliability and Agreement in Measurement
Social Sciences › Communication › News Media Framing and Political Communication
Social Sciences › General Social Sciences › Computational and Text Analysis Methods
Keywords: intercoder reliability, inter-coder agreement, content analysis, mass communication research, reporting practices, percent agreement, reliability index, coding procedures, research guidelines

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Content Analysis in Mass Communication: Assessment and Reporting of Intercoder Reliability

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Citations

Cites

Cited by

Related

Metadata

Record

About these checks