Cited by 3,068Reliability and Agreement in MeasurementNews Media Framing and Political CommunicationComputational and Text Analysis Methods

The DOI matches its Crossref record; 42 of the 42 references it cites were found in a registry or an index.

Content Analysis in Mass Communication: Assessment and Reporting of Intercoder Reliability

Lombard, Matthew, Snyder-Duch, Jennifer, Bracken, Cheryl Campanella·2002·Human Communication Research

DOI 10.1111/j.1468-2958.2002.tb00826.x

Open this paper in CoChat — free account

Abstract

As a method specifically intended for the study of messages, content analysis is fundamental to mass communication research. Intercoder reliability, more specifically termed intercoder agreement, is a measure of the extent to which independent judges make the same coding decisions in evaluating the characteristics of messages, and is at the heart of this method. Yet there are few standard and accessible guidelines available regarding the appropriate procedures to use to assess and report intercoder reliability, or software tools to calculate it. As a result, it seems likely that there is little consistency in how this critical element of content analysis is assessed and reported in published mass communication studies. Following a review of relevant concepts, indices, and tools, a content analysis of 200 studies utilizing content analysis published in the communication literature between 1994 and 1998 is used to characterize practices in the field. The results demonstrate that mass communication researchers often fail to assess (or at least report) intercoder reliability and often rely on percent agreement, an overly liberal index. Based on the review and these results, concrete guidelines are offered regarding procedures for assessment and reporting of this important aspect of content analysis.

Integrity signals

Registry

Crossref · 10.1111/j.1468-2958.2002.tb00826.x

Cites

42 of 42 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

A Coefficient of Agreement for Nominal Scales
Cohen, J. (1960) · Educational and Psychological Measurement · doi:10.1177/001316446002000104 · Cited by 42,398
Content Analysis: An Introduction to its Methodology.
Shelley, M., & Krippendorff, K. (1984) · Journal of the American Statistical Association · doi:10.2307/2288384 · Cited by 24,351
The Content Analysis Guidebook
Neuendorf, K. A. (2017) · doi:10.4135/9781071802878 · Cited by 9,286 · cited 10×
“), but this technique has also been questioned (Neuendorf, 2002). With the coding data in hand, the researcher calculates and reports one or more indices of reliability. Popping (1988) identified 39 different “agreement indices” for coding nominal categories, which excludes several techniques for ratio and interval le…”
BackgroundMethodInfluential
A concordance correlation coefficient to evaluate reproducibility.
Lin, L. I.-K. (1989) · Biometrics · doi:10.2307/2532051 · Cited by 8,785
Weighted kappa: nominal scale agreement provision for scaled disagreement or partial credit
Cohen, J. (1968) · Psychological Bulletin · doi:10.1037/h0026256 · Cited by 8,736
Measuring nominal scale agreement among many raters.
Fleiss, J. L. (1971) · Psychological Bulletin · doi:10.1037/h0031619 · Cited by 8,733
Content Analysis for the Social Sciences and Humanities.
Parker, E. B., & Holsti, O. R. (1970) · American Sociological Review · doi:10.2307/2093233 · Cited by 4,553
Content Analysis in Communication Research.
Pool, I. D. S., & Berelson, B. (1952) · American Sociological Review · doi:10.2307/2088029 · Cited by 3,106
Content Analysis for the Social Sciences and Humanities
Parker, S. R. (1970) · Sociology · doi:10.1177/003803857000400231 · Cited by 2,178
Handbook of Research Methods in Social and Personality Psychology
West, T., Reis, H. T., Judd, C. M., Banaji, M. R., Crandall, C., Giner-Sorolla, R., Biernat, M., Fiedler, K., Ermark, F., Tepper, S. J., et al. (2024) · Cambridge University Press eBooks · doi:10.1017/9781009170123 · Cited by 2,134

Show all 42 references

Research works printed but not linked: 2

Lacy S., 1996, Journalism & Mass Communication Quarterly, 73, 969
Perreault W. D., 1989, Journal of Marketing Research, 26, 135, 10.1177/002224378902600201 · DOI · Paper page

Sorted by citations

Answering the Call for a Standard Reliability Measure for Coding Data
Hayes, A. F., & Krippendorff, K. (2007) · Communication Methods and Measures · doi:10.1080/19312450709336664 · Cited by 4,225
Intercoder Reliability in Qualitative Research: Debates and Practical Guidelines
O’Connor, C., & Joffe, H. (2020) · International Journal of Qualitative Methods · doi:10.1177/1609406919899220 · Cited by 3,871
“Studies that influence important medical, policy, or financial decisions arguably merit a higher ICR threshold than exploratory academic research (Hruschka et al., 2004; Lombard et al., 2002).”
BackgroundMethodInfluential
Coding In-depth Semistructured Interviews: Problems of Unitization and Intercoder Reliability and Agreement
Campbell, J. L., Quincy, C., Osserman, J., & Pedersen, O. K. (2013) · Sociological Methods & Research · doi:10.1177/0049124113500475 · Cited by 2,974
Reliability in content analysis: some common misconceptions and recommendations.
Krippendorff, K. (2004) · Human Communication Research · doi:10.1093/hcr/30.3.411 · Cited by 1,027
Structural Topic Models for Open‐Ended Survey Responses
Roberts, M. E., Stewart, B., Tingley, D. H., Lucas, C., Leder-Luis, J., Gadarian, S. K., Albertson, B. L., & Rand, D. G. (2014) · American Journal of Political Science · doi:10.1111/ajps.12103 · Cited by 2,103
Conducting content‐analysis based literature reviews in supply chain management
Seuring, S. A., & Gold, S. (2012) · Supply Chain Management An International Journal · doi:10.1108/13598541211258609 · Cited by 1,352
Reliability and Inter-rater Reliability in Qualitative Research: Norms and Guidelines for CSCW and HCI Practice
McDonald, N., Schoenebeck, S., & Forte, A. (2019) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3359174 · Cited by 1,206
Content analysis schemes to analyze transcripts of online asynchronous discussion groups: A review
Wever, B. D., Schellens, T., Valcke, M., & Keer, H. V. (2005) · Computers & Education · doi:10.1016/j.compedu.2005.04.005 · Cited by 940
Content Analysis
Drisko, J. W., & Maschi, T. M. (2015) · doi:10.1093/acprof:oso/9780190215491.001.0001 · Cited by 881
A new tool to assess treatment fidelity and evaluation of treatment fidelity across 10 years of health behavior research
Borrelli, B., Sepinwall, D., Ernst, D., Bellg, A. J., Czajkowski, S., Breger, R., DeFrancesco, C., Levesque, C., Sharp, D. L., Ogedegbe, G., et al. (2005) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.73.5.852 · Cited by 803
Reliability in Coding Open-Ended Data: Lessons Learned from HIV Behavioral Research
Hruschka, D. J., Schwartz, D., St.John, D. C., Picone-Decaro, E., Jenkins, R. A., & Carey, J. W. (2004) · Field Methods · doi:10.1177/1525822x04266540 · Cited by 829
Interrater Reliability in Systematic Review Methodology: Exploring Variation in Coder Decision-Making
Belur, J., Tompson, L., Thornton, A., & Simon, M. (2021) · Sociological Methods & Research · doi:10.1177/0049124118799372 · Cited by 753
A systematic approach to conducting review studies: An assessment of content analysis in 25years of IB research
Gaur, A. S., & Kumar, M. (2017) · Journal of World Business · doi:10.1016/j.jwb.2017.11.003 · Cited by 648
Large language models in education: a focus on the complementary relationship between human teachers and ChatGPT
Jeon, J., & Lee, S. (2023) · Education and Information Technologies · doi:10.1007/s10639-023-11834-1 · Cited by 696
Future work selves: How salient hoped-for identities motivate proactive career behaviors.
Strauss, K., Griffin, M. A., & Parker, S. K. (2011) · Journal of Applied Psychology · doi:10.1037/a0026423 · Cited by 686
Preparing, conducting, and analyzing Delphi surveys: Cross-disciplinary practices, new directions, and advancements
Beiderbeck, D., Frevel, N., von der Gracht, H. A., Schmidt, S. L., & Schweitzer, V. M. (2021) · MethodsX · doi:10.1016/j.mex.2021.101401 · Cited by 642
Sample Selection in Systematic Literature Reviews of Management Research
Hiebl, M. R. W. (2023) · Organizational Research Methods · doi:10.1177/1094428120986851 · Cited by 635
Cognitive Processes of Opportunity Recognition: The Role of Structural Alignment
Grégoire, D. A., Barr, P. S., & Shepherd, D. A. (2010) · Organization Science · doi:10.1287/orsc.1090.0462 · Cited by 582
Like, comment, and share on Facebook: How each behavior differs from the other
Kim, C., & Yang, S.-U. (2017) · Public Relations Review · doi:10.1016/j.pubrev.2017.02.006 · Cited by 550
Process guidelines for establishing Intercoder Reliability in qualitative studies
MacPhail, C., Khoza, N., Abler, L., & Ranganathan, M. (2016) · Qualitative Research · doi:10.1177/1468794115577012 · Cited by 534
Media Content Analysis: Its Uses, Benefits and Best Practice Methodology
Macnamara, J. (2005) · UTS ePRESS (University of Technology Sydney) · Cited by 514
Knowledge-based innovation: emergence and embedding of new practice areas in management consulting firms
Anand, N., Gardner, H. K., & Morris, T. (2007) · Academy of Management Journal · doi:10.5465/amj.2007.24634457 · Cited by 509
A team-based approach to open coding: considerations for creating intercoder consensus
Cascio, M. A., Lee, E., Vaudrin, N., & Freedman, D. A. (2019) · Field Methods · doi:10.1177/1525822x19838237 · Cited by 492
A standardized approach to qualitative content analysis of focus group discussions from different countries
Moretti, F., van Vliet, L., Bensing, J. M., Deledda, G., Mazzi, M. A., Rimondini, M., Zimmermann, C., & Fletcher, I. J. (2011) · Patient Education and Counseling · doi:10.1016/j.pec.2011.01.005 · Cited by 453
Qualitative Content Analysis
Forman, J., & Damschroder, L. (2007) · Advances in Bioethics · doi:10.1016/s1479-3709(07)11003-7 · Cited by 808
A River Runs Through It: A Multiple Streams Meta‐Review
Jones, M. D., Peterson, H. L., Pierce, J. J., Herweg, N., Bernal, A., Raney, H. L., & Zahariadis, N. (2016) · Policy Studies Journal · doi:10.1111/psj.12115 · Cited by 399
Environmental Education Research
(2010) · Environmental Education Research · doi:10.1080/13504622.2010.529990 · Cited by 443
A reconceptualization of fear of failure in entrepreneurship
Cacciotti, G., Hayton, J. C., Mitchell, J., & Giazitzoglu, A. (2016) · Journal of Business Venturing · doi:10.1016/j.jbusvent.2016.02.002 · Cited by 492
Evaluating the enablers and barriers for successful implementation of sustainable business practice in ‘lean’ SMEs
Caldera, S., Desha, C., & Dawes, L. A. (2019) · Journal of Cleaner Production · doi:10.1016/j.jclepro.2019.01.239 · Cited by 432
Entrepreneurship Education Research Revisited: The Case of Higher Education
Béchard, J., & Grégoire, D. A. (2005) · Academy of Management Learning and Education · doi:10.5465/amle.2005.16132536 · Cited by 458
Searching for the Good Plan
Berke, P. R., & Godschalk, D. R. (2008) · Journal of Planning Literature · doi:10.1177/0885412208327014 · Cited by 407
Defining service innovation: A review and synthesis
Witell, L., Snyder, H., Gustafsson, A., Fombelle, P. W., & Kristensson, P. (2016) · Journal of Business Research · doi:10.1016/j.jbusres.2015.12.055 · Cited by 418
Application of digital technologies for sustainable product management in a circular economy: A review
Rusch, M., Schöggl, J., & Baumgartner, R. J. (2022) · Business Strategy and the Environment · doi:10.1002/bse.3099 · Cited by 429
SHIFTing artificial intelligence to be responsible in healthcare: a systematic review
Siala, H., & Wang, Y. (2022) · Social Science & Medicine · doi:10.1016/j.socscimed.2022.114782 · Cited by 358
From Words to Numbers: How to Transform Qualitative Data into Meaningful Quantitative Results
Srnka, K. J., & Koeszegi, S. T. (2007) · Schmalenbach Business Review · doi:10.1007/bf03396741 · Cited by 367
Measuring consumer perception of social media marketing activities in e-commerce industry: Scale development & validation
Yadav, M. K., & Rahman, Z. (2017) · Telematics and Informatics · doi:10.1016/j.tele.2017.06.001 · Cited by 352
Agency in Action: Entrepreneurs' Networking Style and Initiation of Economic Exchange
Vissa, B. (2010) · Organization Science · doi:10.1287/orsc.1100.0567 · Cited by 351
The Accuracy of Citizen Science Data: A Quantitative Review
Aceves‐Bueno, E., Adeleye, A. S., Feraud, M., Huang, Y., Tao, M., Yang, Y., & Anderson, S. E. (2017) · Bulletin of the Ecological Society of America · doi:10.1002/bes2.1336 · Cited by 341
Automated content analysis for construction safety: A natural language processing system to extract precursors and outcomes from unstructured injury reports
Tixier, A. J. ‐., Hallowell, M. R., Rajagopalan, B., & Bowman, D. (2015) · Automation in Construction · doi:10.1016/j.autcon.2015.11.001 · Cited by 374
Developing and Testing a Theoretical Framework for Computer‐Mediated Transparency of Local Governments
Grimmelikhuijsen, S., & Welch, E. W. (2012) · Public Administration Review · doi:10.1111/j.1540-6210.2011.02532.x · Cited by 364
Civility 2.0: a comparative analysis of incivility in online political discussion
Rowe, I. (2014) · Information Communication & Society · doi:10.1080/1369118x.2014.940365 · Cited by 338
Drones for parcel and passenger transportation: A literature review
Kellermann, R., Biehle, T., & Fischer, L. (2020) · Transportation Research Interdisciplinary Perspectives · doi:10.1016/j.trip.2019.100088 · Cited by 324
The impacts of an invasive species citizen science training program on participant attitudes, behavior, and science literacy
Crall, A., Jordan, R., Holfelder, K. A., Newman, G., Graham, J., & Waller, D. M. (2012) · Public Understanding of Science · doi:10.1177/0963662511434894 · Cited by 313
An investigation of learners' collaborative knowledge construction performances and behavior patterns in an augmented reality simulation system
Lin, T., Duh, H. B., Li, N., Wang, H.-Y., & Tsai, C. (2013) · Computers & Education · doi:10.1016/j.compedu.2013.05.011 · Cited by 311
Branding co-creation with members of online brand communities
Hajli, N., Shanmugam, M., Papagiannidis, S., Zahay, D. L., & Richard, M. (2016) · Journal of Business Research · doi:10.1016/j.jbusres.2016.08.026 · Cited by 333
How to conduct a Narrative Policy Framework study
Shanahan, E. A., Jones, M. D., & McBeth, M. K. (2018) · The Social Science Journal · doi:10.1016/j.soscij.2017.12.002 · Cited by 362
Cognitive flexibility and adaptive decision‐making: Evidence from a laboratory study of expert decision makers
Martinez, D. L., & Brusoni, S. (2018) · Strategic Management Journal · doi:10.1002/smj.2774 · Cited by 333
The future and social impact of Big Data Analytics in Supply Chain Management: Results from a Delphi study
Roßmann, B., Canzaniello, A., von der Gracht, H. A., & Hartmann, E. (2017) · Technological Forecasting and Social Change · doi:10.1016/j.techfore.2017.10.005 · Cited by 296
The value of social media for innovation: A capability perspective
Muninger, M.-I., Hammedi, W., & Mahr, D. (2019) · Journal of Business Research · doi:10.1016/j.jbusres.2018.10.012 · Cited by 277
Analysis of Five High School Biology Textbooks Used in the United States for Inclusion of the Nature of Science
Chiappetta, E. L., & Fillman, D. A. (2007) · International Journal of Science Education · doi:10.1080/09500690601159407 · Cited by 272
Content Analysis—A Methodological Primer for Gender Research
Neuendorf, K. A. (2010) · Sex Roles · doi:10.1007/s11199-010-9893-0 · Cited by 271
The Multiple Facets of Influence
Dubois, E., & Gaffney, D. (2014) · American Behavioral Scientist · doi:10.1177/0002764214527088 · Cited by 301
Travel photos: Motivations, image dimensions, and affective qualities of places
Pan, S., Lee, J., & Tsai, H. K. (2013) · Tourism Management · doi:10.1016/j.tourman.2013.05.007 · Cited by 266
How organizations framed the 2009 H1N1 pandemic via social and traditional media: Implications for U.S. health communicators
Liu, B. F., & Kim, S. (2011) · Public Relations Review · doi:10.1016/j.pubrev.2011.03.005 · Cited by 253
Is partially automated driving a bad idea? Observations from an on-road study
Banks, V. A., Eriksson, A., O'Donoghue, J., & Stanton, N. A. (2017) · Applied Ergonomics · doi:10.1016/j.apergo.2017.11.010 · Cited by 244
Tweeting as Health Communication: Health Organizations’ Use of Twitter for Health Promotion and Public Engagement
Park, H., Reber, B. H., & Chon, M. (2015) · Journal of Health Communication · doi:10.1080/10810730.2015.1058435 · Cited by 243
Measures of fidelity of delivery of, and engagement with, complex, face‐to‐face health behaviour change interventions: A systematic review of measure quality
Walton, H., Spector, A. E., Tombor, I., & Michie, S. (2017) · British Journal of Health Psychology · doi:10.1111/bjhp.12260 · Cited by 255
Mining Social Media Data for Understanding Students’ Learning Experiences
Chen, X., Vorvoreanu, M., & Madhavan, K. P. C. (2014) · IEEE Transactions on Learning Technologies · doi:10.1109/tlt.2013.2296520 · Cited by 242
Social Construction and Policy Design: A Review of Past Applications
Pierce, J. J., Siddiki, S., Jones, M. D., Schumacher, K., Pattison, A., & Peterson, H. L. (2014) · Policy Studies Journal · doi:10.1111/psj.12040 · Cited by 271
Beyond a coefficient: an interactive process for achieving inter-rater consistency in qualitative coding
Hemmler, V. L., Kenney, A. W., Langley, S. D., Callahan, C. M., Gubbins, E. J., & Holder, S. (2020) · Qualitative Research · doi:10.1177/1468794120976072 · Cited by 241
Gender differences in recommendation letters for postdoctoral fellowships in geoscience
Dutt, K., Pfaff, D. L., Bernstein, A. F., Dillard, J. S., & Block, C. J. (2016) · Nature Geoscience · doi:10.1038/ngeo2819 · Cited by 232
Big Social Data Analytics in Journalism and Mass Communication
Гуо, Л., Vargo, C. J., Pan, Z., Ding, W., & Ishwar, P. (2016) · Journalism & Mass Communication Quarterly · doi:10.1177/1077699016639231 · Cited by 250
Towards integrating construction risk management and stakeholder management: A systematic literature review and future research agendas
Xia, N., Zou, P. X. W., Griffin, M. A., Wang, X., & Zhong, R. (2018) · International Journal of Project Management · doi:10.1016/j.ijproman.2018.03.006 · Cited by 230
Learning presence: Additional research on a new conceptual element within the Community of Inquiry (CoI) framework
Shea, P., Hayes, S., Smith, S. U., Vickers, J. C., Bidjerano, T., Pickett, A., Gozza‐Cohen, M., Wilde, J., & Jian, S.-B. (2011) · The Internet and Higher Education · doi:10.1016/j.iheduc.2011.08.002 · Cited by 251
How the Media Shape Perceptions of Right-Wing Populist Leaders
Bos, L., van der Brug, W., & de Vreese, C. H. (2011) · Political Communication · doi:10.1080/10584609.2011.564605 · Cited by 247
Expediting the Analysis of Qualitative Data in Evaluation
Neal, J. W., Neal, Z. P., VanDyke, E., & Kornbluh, M. (2014) · American Journal of Evaluation · doi:10.1177/1098214014536601 · Cited by 227
What to Do (and Not to Do) with the Comparative Manifestos Project Data
Gemenis, K. (2013) · Political Studies · doi:10.1111/1467-9248.12015 · Cited by 226
Content analysis of e-petitions with topic modeling: How to train and evaluate LDA models?
Hagen, L. (2018) · Information Processing & Management · doi:10.1016/j.ipm.2018.05.006 · Cited by 233
Avoiding Health Information
Barbour, J. B., Rintamaki, L. S., Ramsey, J. A., & Brashers, D. E. (2011) · Journal of Health Communication · doi:10.1080/10810730.2011.585691 · Cited by 222
Plan Quality Evaluation 1994–2012
Lyles, W., & Stevens, M. R. (2014) · Journal of Planning Education and Research · doi:10.1177/0739456x14549752 · Cited by 219
Corporate social responsibility communication of Chinese and global corporations in China
Tang, L., & Li, H. (2009) · Public Relations Review · doi:10.1016/j.pubrev.2009.05.016 · Cited by 218
Inter-Rater Reliability Methods in Qualitative Case Study Research
Cole, R. (2023) · Sociological Methods & Research · doi:10.1177/00491241231156971 · Cited by 294
What's Being Sold and To What End?: A Content Analysis of College Viewbooks
Hartley, M., & Morphew, C. C. (2008) · The Journal of Higher Education · doi:10.1353/jhe.0.0025 · Cited by 212
Goal-oriented requirements engineering: an extended systematic mapping study
Horkoff, J., Aydemir, F. B., Cardoso, E. C. S., Li, T., Maté, A., Paja, E., Salnitri, M., Piras, L., Mylopoulos, J., & Giorgini, P. (2017) · Requirements Engineering · doi:10.1007/s00766-017-0280-z · Cited by 212
Users as innovators in developing countries: The global sources of innovation and diffusion in mobile banking services
van der Boor, P., Oliveira, P., & Veloso, F. M. (2014) · Research Policy · doi:10.1016/j.respol.2014.05.003 · Cited by 209
A linguistic ontology of space for natural language processing
Bateman, J. A., Hois, J., Ross, R., & Tenbrink, T. (2010) · Artificial Intelligence · doi:10.1016/j.artint.2010.05.008 · Cited by 208
Who are you and what do you value? Investigating the role of personality traits and customer-perceived value in online customer engagement
Marbach, J., Lages, C. R., & Nunan, D. (2016) · Journal of Marketing Management · doi:10.1080/0267257x.2015.1128472 · Cited by 223
The Consumer Affinity Construct: Conceptualization, Qualitative Investigation, and Research Agenda
Oberecker, E. M., Riefler, P., & Diamantopoulos, A. (2008) · Journal of International Marketing · doi:10.1509/jimk.16.3.23 · Cited by 204
On pins and needles: How vaccines are portrayed on Pinterest
Guidry, J. P. D., Carlyle, K. E., Messner, M., & Jin, Y. (2015) · Vaccine · doi:10.1016/j.vaccine.2015.08.064 · Cited by 202
The Struggle Over Borders
de Wilde, P., Junk, W. M., & Palmtag, T. (2019) · Cambridge University Press eBooks · doi:10.1017/9781108652698 · Cited by 205
Measuring and Reporting Intercoder Reliability in Plan Quality Evaluation Research
Stevens, M. R., Lyles, W., & Berke, P. R. (2014) · Journal of Planning Education and Research · doi:10.1177/0739456x13513614 · Cited by 197
There and Back Again: A Tale of the Advocacy Coalition Framework
Pierce, J. J., Peterson, H. L., Jones, M. D., Garrard, S. P., & Vu, T. (2017) · Policy Studies Journal · doi:10.1111/psj.12197 · Cited by 198
Content analysis in tourism research
Camprubí, R., & Coromina, L. (2016) · Tourism Management Perspectives · doi:10.1016/j.tmp.2016.03.002 · Cited by 197
Judicial Independence and Political Uncertainty: How the Risk of Override Affects the Court of Justice of the EU
Larsson, O., & Naurin, D. (2016) · International Organization · doi:10.1017/s0020818316000047 · Cited by 195
Australian and U.S. News Media Portrayal of Sharks and Their Conservation
Muter, B. A., Gore, M. L., Gledhill, K. S., Lamont, C., & Huveneers, C. P. M. (2012) · Conservation Biology · doi:10.1111/j.1523-1739.2012.01952.x · Cited by 199
What Creates Interactivity in Online News Discussions? An Exploratory Analysis of Discussion Factors in User Comments on News Items
Ziegele, M., Breiner, T., & Quiring, O. (2014) · Journal of Communication · doi:10.1111/jcom.12123 · Cited by 224
Corporate Social Performance, Firm Size, and Organizational Visibility: Distinct and Joint Effects on Voluntary Sustainability Reporting
Schreck, P., & Raithel, S. (2015) · Business & Society · doi:10.1177/0007650315613120 · Cited by 195
Revisiting the Epistemology of Fact-Checking
Amazeen, M. A. (2015) · Critical Review · doi:10.1080/08913811.2014.993890 · Cited by 192
Teachers' emotions and emotion regulation strategies: Self- and students' perceptions
Jiang, J., Vauras, M., Volet, S., & Wang, Y. (2015) · Teaching and Teacher Education · doi:10.1016/j.tate.2015.11.008 · Cited by 192
Knowledge Exchange and Symbolic Action in Social Media-Enabled Electronic Networks of Practice: A Multilevel Perspective on Knowledge Seekers and Contributors1
Beck, R., Pahlke, I., & Seebach, C. (2014) · MIS Quarterly · doi:10.25300/misq/2014/38.4.14 · Cited by 186
The role of academic motivation in Computer-Supported Collaborative Learning
Rienties, B., Tempelaar, D. T., den Bossche, P. V., Gijselaers, W., & Segers, M. S. R. (2009) · Computers in Human Behavior · doi:10.1016/j.chb.2009.05.012 · Cited by 198
The relationship between sustainability and customer satisfaction in hospitality: An explorative investigation using eWOM as a data source
Gerdt, S., Wagner, E., & Schewe, G. (2019) · Tourism Management · doi:10.1016/j.tourman.2019.02.010 · Cited by 185
Understanding a Period of Policy Change: The Case of Hydraulic Fracturing Disclosure Policy in C olorado
Heikkila, T., Pierce, J. J., Gallaher, S., Kagan, J. A., Crow, D. A., & Weible, C. M. (2014) · Review of Policy Research · doi:10.1111/ropr.12058 · Cited by 184
Six decades of project management research: Thematic trends and future opportunities
Padalkar, M., & Gopinath, S. (2016) · International Journal of Project Management · doi:10.1016/j.ijproman.2016.06.006 · Cited by 206
Assessing the Reliability and Validity of Expert Interviews
Dorussen, H., Lenz, H., & Blavoukos, S. (2005) · European Union Politics · doi:10.1177/1465116505054835 · Cited by 215
An experiential model of consumer engagement in social media
Tafesse, W. (2016) · Journal of Product & Brand Management · doi:10.1108/jpbm-05-2015-0879 · Cited by 200
Studies of user-generated content: A systematic review
Naab, T. K., & Sehl, A. (2017) · Journalism · doi:10.1177/1464884916673557 · Cited by 151
Issue Importance as a Moderator of Framing Effects
Lecheler, S. K., de Vreese, C. H., & Slothuus, R. (2009) · Communication Research · doi:10.1177/0093650209333028 · Cited by 190
Stigma among patients with lung cancer: a patient-reported measurement model
Hamann, H. A., Ostroff, J. S., Marks, E. G., Gerber, D. E., Schiller, J. H., & Lee, S. J. C. (2013) · Psycho-Oncology · doi:10.1002/pon.3371 · Cited by 185
Why Individuals Participate in Micro-task Crowdsourcing Work Environment: Revealing Crowdworkers’ Perceptions
Deng, X. N., & Joshi, K. D. (2016) · Journal of the Association for Information Systems · doi:10.17705/1jais.00441 · Cited by 175

Authors: Lombard, M., Snyder-Duch, J., & Bracken, C. C.
Affiliations: Temple University (US)
Cleveland State University (US)
Venue: Human Communication Research
Publisher: Oxford University Press (OUP)
Published: 2002-10
Volume: 28
Issue: 4
Pages: 587-604
ISSN: 0360-3989 (ISSN-L)
1468-2958
DOI: 10.1111/j.1468-2958.2002.tb00826.x
OpenAlex: W2160375265
Cited by: 3,068
References: 42
Citations by year: 2026: 234 · 2025: 173 · 2024: 136 · 2023: 143 · 2022: 212 · 2021: 238 · 2020: 206 · 2019: 206 · 2018: 174 · 2017: 168 · 2016: 174 · 2015: 153 · 2014: 168 · 2013: 158 · 2012: 117
FWCI: 19.75
Type: journal article
Language: en
Open access: closed
Licence: doi.wiley.com/10.1002/tdm_license_1.1 (tdm, 4718 d)
Locations: Human Communication Research (publishedVersion)
EngagedScholarship @ Cleveland State University (Cleveland State University) (submittedVersion)
Topics: Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Reliability and Agreement in Measurement
Social Sciences › Communication › News Media Framing and Political Communication
Social Sciences › General Social Sciences › Computational and Text Analysis Methods
Keywords: intercoder reliability, inter-coder agreement, content analysis, mass communication research, reporting practices, percent agreement, reliability index, coding procedures, research guidelines

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Content Analysis in Mass Communication: Assessment and Reporting of Intercoder Reliability

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Citations

Cites

Cited by

Related

Metadata

Record

About these checks