How Measurement Error in Content Analysis and Self-Reported Media Use Leads to Minimal Media Effect Findings in Linkage Analyses: A Simulation Study
The DOI matches its Crossref record; 7 of the 7 references it cites were found in a registry or an index.
Abstract
In the debate on minimal media effects and their causes, methodological concerns about measurement are rarely discussed. We argue that even in state-of-the-art media-effects studies that combine measures of media messages and media use (i.e., linkage analyses), measurement error in both the media content analysis and the media use self-reports will typically lead to severely downward-biased effect estimates. We demonstrate this phenomenon using a large Monte Carlo simulation with varying parameters of the content analysis and the survey study. Results show that measurement error in the content analysis and media use variables does indeed lead to smaller effect estimates, especially when the media messages of interest are relatively rare. We discuss these findings as well as possible remedies and implications for future research.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Authentic leadership development: Getting to the root of positive forms of leadership
Avolio, B. J., & Gardner, W. L. (2005) · The Leadership Quarterly · doi:10.1016/j.leaqua.2005.03.001 · Cited by 5,478
A New Era of Minimal Effects? The Changing Foundations of Political Communication
Bennett, W. L., & Iyengar, S. (2008) · Journal of Communication · doi:10.1111/j.1460-2466.2008.00410.x · Cited by 1,558
The Immensely Inflated News Audience: Assessing Bias in Self-Reported News Exposure
Prior, M. (2009) · Public Opinion Quarterly · doi:10.1093/poq/nfp002 · Cited by 561
The Accuracy of Self-Reported Internet Use—A Validation Study Using Client Log Data
Scharkow, M. (2016) · Communication Methods and Measures · doi:10.1080/19312458.2015.1118446 · Cited by 408 · cited 2×
“Comparing self-reports with passive measures, frequent overreporting was recently found by Wonneberger and Irazoqui (2016) for general television use and by Scharkow (2016) for internet use.”
BackgroundSelf-citationTreating Words as Data with Error: Uncertainty in Text Statements of Policy Positions
Benoit, K., Laver, M., & Mikhaylov, S. (2009) · American Journal of Political Science · doi:10.1111/j.1540-5907.2009.00383.x · Cited by 356 · cited 1×
“One flexible and powerful approach for error correction is simulation-extrapolation (Cook & Stefanski, 1994), which has been successfully applied to both survey and content analysis data (Benoit, Laver, & Mikhaylov, 2009), but not systematically evaluated for linkage analyses.”
MethodSchemer, C. (2012) · Journal of Communication · doi:10.1111/j.1460-2466.2012.01672.x · Cited by 281 · cited 2×
“Swiss referendum campaigns were the contexts for studies of media effects on stereotypic attitudes toward immigrants (Schemer, 2012) and on the timing of voting decisions (Matthes, 2012).”
BackgroundComputer-Assisted Topic Classification for Mixed-Methods Social Science Research
Hillard, D., Purpura, S., & Wilkerson, J. (2008) · Journal of Information Technology & Politics · doi:10.1080/19331680801975367 · Cited by 138
Cited by
Ernala, S. K., Burke, M., Leavitt, A., & Ellison, N. B. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376435 · Cited by 179
“Historically, self-report validity is low for exposure to television and newspapers [3, 4, 44, 78], general media use [16, 62], and news access [70, 43, 32, 73, 56].”
BackgroundThe Effects of Right-Wing Populist Communication on Emotions and Cognitions toward Immigrants
Wirz, D. S., Wettstein, M., Schulz, A., Müller, P., Schemer, C., Ernst, N., Esser, F., & Wirth, W. (2018) · The International Journal of Press/Politics · doi:10.1177/1940161218788956 · Cited by 145
Song, H., Tolochko, P., Eberl, J.-M., Eisele, O., Greussing, E., Heidenreich, T., Lind, F., Galyga, S., & Boomgaarden, H. G. (2020) · Political Communication · doi:10.1080/10584609.2020.1723752 · Cited by 123
Fikkers, K. M., Piotrowski, J. T., & Valkenburg, P. M. (2019) · Journal of Communication · doi:10.1093/joc/jqz003 · Cited by 31
Embedding Democratic Values into Social Media AIs via Societal Objective Functions
Jia, C., Lam, M. S., Mai, M. C., Hancock, J. T., & Bernstein, M. S. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3641002 · Cited by 31
Metadata
- Authors
- Scharkow, M., & Bachl, M.
- Venue
- Political Communication
- Publisher
- Informa UK Limited
- Published
- 2016-10-31
- Volume
- 34
- Issue
- 3
- Pages
- 323-343
- ISSN
- 1058-4609 (ISSN-L)
- 1091-7675
- OpenAlex
- W2543606719
- Cited by
- 100
- References
- 7
- FWCI
- 30.39
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: How Measurement Error in Content Analysis and Self-Reported Media Use Leads to Minimal Media Effect Findings in Linkage Analyses: A Simulation Study. https://app.cochat.ai/papers/10.1080/10584609.2016.1235640
- The paper
- Scharkow, M., & Bachl, M. (2017). How Measurement Error in Content Analysis and Self-Reported Media Use Leads to Minimal Media Effect Findings in Linkage Analyses: A Simulation Study. Political Communication. https://doi.org/10.1080/10584609.2016.1235640
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.