The DOI matches its Crossref record; 20 of the 20 references it cites were found in a registry or an index.
Content-Analysis Research: An Examination of Applications with Directives for Improving Research Reliability and Objectivity
Abstract
This article provides an empirical review and synthesis of published studies that have used content-analysis methods. Harold Kassarjian's critical guidelines for content-analysis research were used to examine the methods employed in 128 studies. The guidelines were expanded by providing an empirical investigation of multiple dimensions of objectivity. Reliability issues were also assessed by examining factors central to the replication and interjudge coefficient calculations. The findings indicate a general need for improvement in the application of content-analysis methods. Suggestions for calculating reliability coefficients and for improving the objectivity and reliability of research are offered.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
A Coefficient of Agreement for Nominal Scales
Cohen, J. (1960) · Educational and Psychological Measurement · doi:10.1177/001316446002000104 · Cited by 42,398
Statistical principles in experimental design.
Winer, B. J. (1962) · McGraw-Hill Book Company · doi:10.1037/11774-000 · Cited by 26,755
Content Analysis: An Introduction to its Methodology.
Shelley, M., & Krippendorff, K. (1984) · Journal of the American Statistical Association · doi:10.2307/2288384 · Cited by 24,312
Weber, R. (1990) · doi:10.4135/9781412983488 · Cited by 7,750
Foundations of Behavioral Research
Stockton, R., & Kerlinger, F. N. (1974) · American Educational Research Journal · doi:10.2307/1162206 · Cited by 7,347
Content Analysis for the Social Sciences and Humanities.
Parker, E. B., & Holsti, O. R. (1970) · American Sociological Review · doi:10.2307/2093233 · Cited by 4,564
Comparative Politics and the Comparative Method
Lijphart, A. (1971) · American Political Science Review · doi:10.2307/1955513 · Cited by 3,151
Content Analysis in Consumer Research
Kassarjian, H. H. (1977) · Journal of Consumer Research · doi:10.1086/208674 · Cited by 2,303
Reliability of Content Analysis: The Case of Nominal Scale Coding
Scott, W. A. (1955) · Public Opinion Quarterly · doi:10.1086/266577 · Cited by 1,853
Multimethod Research: A Synthesis of Styles
Brewer, J., & Hunter, A. (1989) · Cited by 1,821
Cited by
Intercoder Reliability in Qualitative Research: Debates and Practical Guidelines
O’Connor, C., & Joffe, H. (2020) · International Journal of Qualitative Methods · doi:10.1177/1609406919899220 · Cited by 3,871
“Some researchers, whose analytic approaches prioritize increasing the coding frame’s external objectivity, recommend coders should be people external to the research team who had no role in designing the coding frame (Kolbe & Burnett, 1991).”
BackgroundMethodContent Analysis in Mass Communication: Assessment and Reporting of Intercoder Reliability
Lombard, M., Snyder-Duch, J., & Bracken, C. C. (2002) · Human Communication Research · doi:10.1111/j.1468-2958.2002.tb00826.x · Cited by 3,068
The Effects of Strategic Orientations on Technology- and Market-Based Breakthrough Innovations
Zhou, K. Z., Yim, C. K. (., & Tse, D. K. (2005) · Journal of Marketing · doi:10.1509/jmkg.69.2.42.60756 · Cited by 1,778
Exploring the reliability of social and environmental disclosures content analysis
Milne, M. J., & Adler, R. W. (1999) · Accounting, Auditing & Accountability Journal · doi:10.1108/09513579910270138 · Cited by 1,425
Theorising from case studies: Towards a pluralist future for international business research
Welch, C., Piekkari, R., Plakoyiannaki, E., & Paavilainen-Mäntymäki, E. (2011) · Journal of International Business Studies · doi:10.1057/jibs.2010.55 · Cited by 1,316
Homburg, C., & Pflesser, C. (2000) · Journal of Marketing Research · doi:10.1509/jmkr.37.4.449.18786 · Cited by 1,282
Conducting content‐analysis based literature reviews in supply chain management
Seuring, S., & Gold, S. (2012) · Supply Chain Management: An International Journal · doi:10.1108/13598541211258609 · Cited by 1,245
International Entrepreneurship research (1989–2009): A domain ontology and thematic analysis
Jones, M. V., Coviello, N., & Tang, Y. K. (2011) · Journal of Business Venturing · doi:10.1016/j.jbusvent.2011.04.001 · Cited by 1,241
Supply chain resilience in the global financial crisis: an empirical study
Jüttner, U., & Maklan, S. (2011) · Supply Chain Management: An International Journal · doi:10.1108/13598541111139062 · Cited by 1,213
The Critical Incident Technique in Service Research
Gremler, D. D. (2004) · Journal of Service Research · doi:10.1177/1094670504266138 · Cited by 1,078
Kleinheksel, A. J., Rockich-Winston, N., Tawfik, H., & Wyatt, T. R. (2020) · American Journal of Pharmaceutical Education · doi:10.5688/ajpe7113 · Cited by 1,054
How Does Brand-related User-generated Content Differ across YouTube, Facebook, and Twitter?
Smith, A. N., Fischer, E., & Yongjian, C. (2012) · Journal of Interactive Marketing · doi:10.1016/j.intmar.2012.01.002 · Cited by 1,044
Best Practices in Quantitative Methods
Osborne, J. (2008) · Sage eBooks · doi:10.4135/9781412995627 · Cited by 942
Pavlou, P. A., & Dimoka, A. (2006) · Information Systems Research · doi:10.1287/isre.1060.0106 · Cited by 930
Sustainable supply chain management and inter‐organizational resources: a literature review
Gold, S., Seuring, S., & Beske, P. (2010) · Corporate Social Responsibility and Environmental Management · doi:10.1002/csr.207 · Cited by 918
Optimization models in emergency logistics: A literature review
Caunhye, A. M., Nie, X., & Pokharel, S. (2012) · Socio-Economic Planning Sciences · doi:10.1016/j.seps.2011.04.004 · Cited by 854
Reliability in Coding Open-Ended Data: Lessons Learned from HIV Behavioral Research
Hruschka, D. J., Schwartz, D., St.John, D. C., Picone-Decaro, E., Jenkins, R. A., & Carey, J. W. (2004) · Field Methods · doi:10.1177/1525822x04266540 · Cited by 829
Grove, S. J., & Fisk, R. P. (1997) · Journal of Retailing · doi:10.1016/s0022-4359(97)90015-4 · Cited by 764
Belur, J., Tompson, L., Thornton, A., & Simon, M. (2021) · Sociological Methods & Research · doi:10.1177/0049124118799372 · Cited by 753
On Product Uncertainty in Online Markets: Theory and Evidence1
Dimoka, A., Hong, Y., & Pavlou, P. A. (2012) · MIS Quarterly · doi:10.2307/41703461 · Cited by 682
CORPORATE SOCIAL RESPONSIBILITY REPORTS: A THEMATIC ANALYSIS RELATED TO SUPPLY CHAIN MANAGEMENT
TATE, W. L., ELLRAM, L. M., & KIRCHOFF, J. F. (2010) · Journal of Supply Chain Management · doi:10.1111/j.1745-493x.2009.03184.x · Cited by 679
Cao, Q., Duan, W., & Gan, Q. (2011) · Decision Support Systems · doi:10.1016/j.dss.2010.11.009 · Cited by 640
Political Polarization on Twitter
Conover, M., Ratkiewicz, J., Francisco, M., Goncalves, B., Menczer, F., & Flammini, A. (2021) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v5i1.14126 · Cited by 992
“…annotations agree with an objective judge is Cohen's Kappa, defined as κ = P (α) − P () 1 − P () (2) where P (α) is the observed rate of agreement between annotators , and P () is the expected rate of random agreement given the relative frequency of each class label (Krippendorff 2004; Kolbe 1991).”
BackgroundMethodInfluentialBorn Unequal: A Study of the Helpfulness of User-Generated Product Reviews
Pan, Y., & Zhang, J. Q. (2011) · Journal of Retailing · doi:10.1016/j.jretai.2011.05.002 · Cited by 610
A Content Analysis of Environmental Advertising Claims: A Matrix Method Approach
Carlson, L., Grove, S. J., & Kangun, N. (1993) · Journal of Advertising · doi:10.1080/00913367.1993.10673409 · Cited by 602
Yawar, S. A., & Seuring, S. (2017) · Journal of Business Ethics · doi:10.1007/s10551-015-2719-9 · Cited by 580
An exploration of consumer resistance to innovation and its antecedents
Kleijnen, M., Lee, N., & Wetzels, M. (2009) · Journal of Economic Psychology · doi:10.1016/j.joep.2009.02.004 · Cited by 574
Assessing Performance Outcomes in Marketing
Katsikeas, C. S., Morgan, N. A., Leonidou, L. C., & Hult, G. T. M. (2016) · Journal of Marketing · doi:10.1509/jm.15.0287 · Cited by 559
What Does the Brain Tell Us About Trust and Distrust? Evidence from a Functional Neuroimaging Study
Dimoka, A. (2010) · MIS Quarterly · doi:10.2307/20721433 · Cited by 550
Megamarketing: The Creation of Markets as a Social Process
Humphreys, A. (2010) · Journal of Marketing · doi:10.1509/jm.74.2.1 · Cited by 543
An assessment of the measurement of performance in international business research
Hult, G. T. M., Ketchen, D. J., Griffith, D. A., Chabowski, B. R., Hamman, M. K., Dykes, B. J., Pollitte, W. A., & Cavusgil, S. T. (2008) · Journal of International Business Studies · doi:10.1057/palgrave.jibs.8400398 · Cited by 487
Shades of Green: A Multidimensional Analysis of Environmental Advertising
Banerjee, S., Gulas, C. S., & Iyer, E. (1995) · Journal of Advertising · doi:10.1080/00913367.1995.10673473 · Cited by 486
The role of strategic alliances in high‐technology new product development
Kotabe, M., & Scott Swan, K. (1995) · Strategic Management Journal · doi:10.1002/smj.4250160804 · Cited by 429
Service experience co-creation: conceptualization, implications, and future research directions
Jaakkola, E., Helkkula, A., & Aarikka-Stenroos, L. (2015) · Journal of Service Management · doi:10.1108/josm-12-2014-0323 · Cited by 423
Building, measuring, and profiting from customer loyalty
Watson, G. F., Beck, J. T., Henderson, C. M., & Palmatier, R. W. (2015) · Journal of the Academy of Marketing Science · doi:10.1007/s11747-015-0439-4 · Cited by 375
Influence of Facebook brand-page posts on online engagement
Luarn, P., Lin, Y.-F., & Chiu, Y.-P. (2015) · Online Information Review · doi:10.1108/oir-01-2015-0029 · Cited by 370
Kandampully, J., Zhang, T., & Jaakkola, E. (2018) · International Journal of Contemporary Hospitality Management · doi:10.1108/ijchm-10-2015-0549 · Cited by 357
Business Advertising Appeals as a Mirror of Cultural Dimensions: A Study of Eleven Countries
Albers-Miller, N. D., & Gelb, B. D. (1996) · Journal of Advertising · doi:10.1080/00913367.1996.10673512 · Cited by 357
Relationship marketing defined? An examination of current relationship marketing definitions
Harker, M. J. (1999) · Marketing Intelligence & Planning · doi:10.1108/02634509910253768 · Cited by 353
Han, H. (2020) · Business Strategy and the Environment · doi:10.1002/bse.2545 · Cited by 350
Mixed methods designs in marketing research
Harrison, R. L., & Reilly, T. M. (2011) · Qualitative Market Research: An International Journal · doi:10.1108/13522751111099300 · Cited by 331
Ahuvia, A. (2001) · Social Indicators Research · doi:10.1023/a:1011087813505 · Cited by 304
Methodological Rigor in Mixed Methods: An Application in Management Studies
Harrison, R. L., Reilly, T. M., & Creswell, J. W. (2020) · Journal of Mixed Methods Research · doi:10.1177/1558689819900585 · Cited by 303
A content analysis of research approaches in logistics research
Spens, K. M., & Kovács, G. (2006) · International Journal of Physical Distribution & Logistics Management · doi:10.1108/09600030610676259 · Cited by 299
Characterising the concept of service experience
Helkkula, A. (2011) · Journal of Service Management · doi:10.1108/09564231111136872 · Cited by 295
Asian-Americans: Television Advertising and the “Model Minority” Stereotype
Taylor, C. R., & Stern, B. B. (1997) · Journal of Advertising · doi:10.1080/00913367.1997.10673522 · Cited by 290
(2012) · Cited by 289
Revisiting Consumer Ethnocentrism: Review, Reconceptualization, and Empirical Testing
Siamagka, N.-T., & Balabanis, G. (2015) · Journal of International Marketing · doi:10.1509/jim.14.0085 · Cited by 289
Lumineau, F., & Henderson, J. E. (2012) · Journal of Operations Management · doi:10.1016/j.jom.2012.03.005 · Cited by 284
Local Open Innovation: A means for public policy to increase collaboration for innovation in SMEs
Leckel, A., Veilleux, S., & Dana, L. P. (2020) · Technological Forecasting and Social Change · doi:10.1016/j.techfore.2019.119891 · Cited by 280
Content Analysis—A Methodological Primer for Gender Research
Neuendorf, K. A. (2011) · Sex Roles · doi:10.1007/s11199-010-9893-0 · Cited by 272
Indexicality and the Verification Function of Irreplaceable Possessions: A Semiotic Analysis
Grayson, K., & Shulman, D. (2000) · Journal of Consumer Research · doi:10.1086/314306 · Cited by 271
Expectation Processes in Satisfaction Formation: A Field Study
Oliver, R. L., & Burke, R. R. (1999) · Journal of Service Research · doi:10.1177/109467059913002 · Cited by 271
Análisis cualitativo de contenido: una alternativa metodológica alcanzable
Cáceres, P. (2008) · Psicoperspectivas. Individuo y Sociedad · doi:10.5027/psicoperspectivas-vol2-issue1-fulltext-3 · Cited by 269
(2012) · Research in Consumer Behavior · doi:10.1108/s0885-2111(2012)0000014024 · Cited by 268
From “information” to “knowing”: Exploring the role of social media in contemporary news consumption
Pentina, I., & Tarafdar, M. (2014) · Computers in Human Behavior · doi:10.1016/j.chb.2014.02.045 · Cited by 259
Frosch, D. L., Krueger, P. M., Hornik, R. C., Cronholm, P. F., & Barg, F. K. (2007) · The Annals of Family Medicine · doi:10.1370/afm.611 · Cited by 256
Backshoring strategy and the adoption of Industry 4.0: Evidence from Europe
Ancarani, A., Di Mauro, C., & Mascali, F. (2019) · Journal of World Business · doi:10.1016/j.jwb.2019.04.003 · Cited by 248
Partisan asymmetries in online political activity
Conover, M. D., Gonçalves, B., Flammini, A., & Menczer, F. (2012) · EPJ Data Science · doi:10.1140/epjds6 · Cited by 234
Marketing images of gender: A visual analysis
Schroeder, J. E., & Borgerson, J. L. (1998) · Consumption Markets & Culture · doi:10.1080/10253866.1998.9670315 · Cited by 188
The Use of Humor to Mask Deceptive Advertising: It's No Laughing Matter
Shabbir, H., & Thwaites, D. (2007) · Journal of Advertising · doi:10.2753/joa0091-3367360205 · Cited by 123
Meiklejohn, S., Ryan, L., & Palermo, C. (2016) · Journal of Nutrition Education and Behavior · doi:10.1016/j.jneb.2016.07.015 · Cited by 120
R.J. Reynolds’ targeting of African Americans: 1988–2000
Balbach, E. D., Gasior, R. J., & Barbeau, E. M. (2003) · American Journal of Public Health · doi:10.2105/ajph.93.5.822 · Cited by 115
Credibility of information in online communities
Hajli, M. N., Sims, J., Featherman, M., & Love, P. E. D. (2015) · Journal of Strategic Marketing · doi:10.1080/0965254x.2014.920904 · Cited by 109
Making sense of text: artificial intelligence-enabled content analysis
Lee, L. W., Dabirian, A., McCarthy, I. P., & Kietzmann, J. (2020) · European Journal of Marketing · doi:10.1108/ejm-02-2019-0219 · Cited by 103
Khan, Z., Lew, Y. K., & Sinkovics, R. R. (2015) · Global Strategy Journal · doi:10.1002/gsj.1089 · Cited by 61
Searching the web for global brands: how American brands standardise their web sites in Europe
Okazaki, S. (2005) · European Journal of Marketing · doi:10.1108/03090560510572034 · Cited by 59
Analysing tourism slogans in top tourism destinations
Galí, N., Camprubí, R., & Donaire, J. A. (2017) · Journal of Destination Marketing & Management · doi:10.1016/j.jdmm.2016.04.004 · Cited by 53
Assessing the Greenness of Environmental Advertising Claims Made by Multinational Industrial Firms
Leonidou, L. C., Leonidou, C. N., Hadjimarcou, J. S., & Lytovchenko, I. (2014) · Industrial Marketing Management · doi:10.1016/j.indmarman.2014.02.003 · Cited by 50
Sustainable Supplier Evaluation: From Current Criteria to Reconstruction Based on ESG Requirements
Lou, S., You, X., & Xu, T. (2024) · Sustainability · doi:10.3390/su16020757 · Cited by 26
Methodische Ansätze zu Evidenzsynthesen in der Bildungstechnologie: Eine tertiäre Übersichtsarbeit
Buntins, K., Bedenlier, S., Marín, V., Händel, M., & Bond, M. (2023) · MedienPädagogik: Zeitschrift für Theorie und Praxis der Medienbildung · doi:10.21240/mpaed/54/2023.12.20.x · Cited by 13
Content Analysis Using Specific Natural Language Processing Methods for Big Data
Pirnau, M., Botezatu, M. A., Priescu, I., Hosszu, A., Tabusca, A., Coculescu, C., & Oncioiu, I. (2024) · Electronics · doi:10.3390/electronics13030584 · Cited by 11
Metadata
- Authors
- Kolbe, R. H., & Burnett, M. S.
- Venue
- Journal of Consumer Research
- Publisher
- Oxford University Press (OUP)
- Published
- 1991-09
- Volume
- 18
- Issue
- 2
- Pages
- 243
- ISSN
- 0093-5301 (ISSN-L)
- 1537-5277
- OpenAlex
- W2035040004
- Cited by
- 1,520
- References
- 20
- FWCI
- 14.60
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.