The DOI matches its Crossref record; 51 of the 52 references it cites were found in a registry or an index.
Fair and Balanced? Quantifying Media Bias through Crowdsourced Content Analysis
Open accessAbstract
It is widely thought that news organizations exhibit ideological bias, but rigorously quantifying such slant has proven methodologically challenging. Through a combination of machine learning and crowdsourcing techniques, we investigate the selection and framing of political issues in 15 major U.S. news outlets. Starting with 803,146 news stories published over 12 months, we first used supervised learning algorithms to identify the 14% of articles pertaining to political events. We then recruited 749 online human judges to classify a random subset of 10,950 of these political articles according to topic and ideological position. Our analysis yields an ideological ordering of outlets consistent with prior work. We find, however, that news outlets are considerably more similar than generally believed. Specifically, with the exception of political scandals, we find that major news organizations present topics in a largely non-partisan manner, casting neither Democrats nor Republicans in a particularly favorable or unfavorable light. Moreover, again with the exception of political scandals, there is little evidence of systematic differences in story selection, with all major news outlets covering a wide variety of topics with frequency largely unrelated to the outlet's ideological position. Finally, we find that news organizations express their ideological bias not by directly advocating for a preferred political party, but rather by disproportionately criticizing one side, a con…
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“In contrast to our multi-dimensional approach that examines the ideology of the issue being discussed instead of the author stance, much of the recent work in computational linguistics has been dedicated to the latter (detection of ideological bias in news media) while collapsing ideology to one dimension”
From Political Ideology and Polarization: A Multi-dimensional Approach (2022) · cites this paper as Budak et al., 2016
Citations
Cites
Cortes, C., & Vapnik, V. (1995) · Machine Learning · doi:10.1007/bf00994018 · Cited by 59,421
(2003) · Journal of Machine Learning Research · doi:10.5555/944919.944937 · Cited by 32,337
Amazon's Mechanical Turk: A New Source of Inexpensive, Yet High-Quality, Data?
Buhrmester, M., Kwang, T., & Gosling, S. D. (2011) · Perspectives on Psychological Science · doi:10.1177/1745691610393980 · Cited by 9,996
The Agenda-Setting Function of Mass Media
McCombs, M. E., & Shaw, D. L. (1972) · Public Opinion Quarterly · doi:10.1086/267990 · Cited by 9,537
Large-Scale Machine Learning with Stochastic Gradient Descent
Bottou, L. (2010) · Proceedings of COMPSTAT'2010 · doi:10.1007/978-3-7908-2604-3_16 · Cited by 5,795
Media Discourse and Public Opinion on Nuclear Power: A Constructionist Approach
Gamson, W. A., & Modigliani, A. (1989) · American Journal of Sociology · doi:10.1086/229213 · Cited by 4,997
(2009) · Minds at UW (University of Wisconsin) · Cited by 4,791
A generalization of sampling without replacement from a finite universe
Horvitz, D. G., & Thompson, D. J. (1952) · Journal of the American Statistical Association · doi:10.1080/01621459.1952.10483446 · Cited by 4,293
Is anyone responsible? How television frames political issues.
Iyengar, S. (1991) · University of Chicago Press eBooks · doi:10.7208/chicago/9780226388533.001.0001 · Cited by 4,258
Evaluating Online Labor Markets for Experimental Research: Amazon.com's Mechanical Turk
Berinsky, A. J., Huber, G. A., & Lenz, G. S. (2012) · Political Analysis · doi:10.1093/pan/mpr057 · Cited by 4,008
Research works printed but not linked: 6
- 10.1162/jmlr.2003.3.4–5.993 · DOI
- Flaxman Seth Goel Sharad Rao Justin M. . 2015. “Ideological Segregation and the Effects of Social Media on News Consumption.” Social Science Research Network 2363701. Available at http://ssrn.com/abstract=2363701.
- Gamson William A . 1992. Talking Politics . Cambridge: Cambridge University Press.
- Kohut Andrew Doherty Carroll Dimock Michael Keeter Scott . 2012. “In CNews Landscape, Even Television Is Vulnerable.” Pew Center for the People and the Press. Available at http://www.people-press.org/2012/09/27/section-4-demographics-and-political-views-of-news-audiences/.
- Sutter, 2001, “Can the Media Be So Liberal? The Economics of Media Bias.”, Cato Journal, 20, 431
- Wais Paul Lingamneni Shivaram Cook Duncan Fennell Jason Goldenberg Benjamin Lubarov Daniel Marin David Simons Hari . 2010. “Towards Building a High-Quality Workforce with Mechanical Turk.” Proceedings of Computational Social Science and the Wisdom of Crowds, NIPS, 1–5.
Cited by
Sorted by citations
Exposure to ideologically diverse news and opinion on Facebook
Bakshy, E., Messing, S., & Adamic, L. A. (2015) · Science · doi:10.1126/science.aaa1160 · Cited by 3,245
Fake news on Twitter during the 2016 U.S. presidential election
Grinberg, N., Joseph, K., Friedland, L., Swire-Thompson, B., & Lazer, D. (2019) · Science · doi:10.1126/science.aau2706 · Cited by 1,803
Frenemies: How Social Media Polarizes America
Settle, J. E. (2018) · doi:10.1017/9781108560573 · Cited by 377
(Almost) Everything in Moderation: New Evidence on Americans' Online Media Diets
Guess, A. M. (2021) · American Journal of Political Science · doi:10.1111/ajps.12589 · Cited by 342
Asymmetric ideological segregation in exposure to political news on Facebook
González-Bailón, S., Lazer, D., Barberá, P., Zhang, M., Allcott, H., Brown, T., Crespo-Tenorio, A., Freelon, D., Gentzkow, M., Guess, A. M., et al. (2023) · Science · doi:10.1126/science.ade7138 · Cited by 315
Metadata
- Authors
- Budak, C., Goel, S., & Rao, J. M.
- Affiliations
- Microsoft (United States) (US)
- Corresponding author
- Budak, Ceren
- Venue
- Public Opinion Quarterly
- Publisher
- Oxford University Press (OUP)
- Published
- 2016
- Volume
- 80
- Issue
- S1
- Pages
- 250-271
- ISSN
- 0033-362X (ISSN-L)
- 1537-5331
- OpenAlex
- W1119807432
- Cited by
- 336
- References
- 52
- Citations by year
- 2026: 18 · 2025: 40 · 2024: 28 · 2023: 38 · 2022: 30 · 2021: 55 · 2020: 38 · 2019: 39 · 2018: 32 · 2017: 9 · 2016: 5
- FWCI
- 81.00
- Type
- journal article
- Language
- en
- Open access
- green
- doi.org
- Locations
- Public Opinion Quarterly (publishedVersion)
- SSRN Electronic Journal (acceptedVersion)
- HighWire Press Open Archive (submittedVersion)
- Topics
- Social Sciences › Sociology and Political Science › Media Influence and Politics
- Social Sciences › Communication › News Media Framing and Political Communication
- Social Sciences › General Social Sciences › Computational and Text Analysis Methods
- Keywords
- media bias, ideological bias, news framing, media coverage of politics, partisan bias, political scandal, news selection
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
