Fair and Balanced? Quantifying Media Bias through Crowdsourced Content Analysis
Open accessAbstract
It is widely thought that news organizations exhibit ideological bias, but rigorously quantifying such slant has proven methodologically challenging. Through a combination of machine learning and crowdsourcing techniques, we investigate the selection and framing of political issues in 15 major U.S. news outlets. Starting with 803,146 news stories published over 12 months, we first used supervised learning algorithms to identify the 14% of articles pertaining to political events. We then recruited 749 online human judges to classify a random subset of 10,950 of these political articles according to topic and ideological position. Our analysis yields an ideological ordering of outlets consistent with prior work. We find, however, that news outlets are considerably more similar than generally believed. Specifically, with the exception of political scandals, we find that major news organizations present topics in a largely non-partisan manner, casting neither Democrats nor Republicans in a particularly favorable or unfavorable light. Moreover, again with the exception of political scandals, there is little evidence of systematic differences in story selection, with all major news outlets covering a wide variety of topics with frequency largely unrelated to the outlet's ideological position. Finally, we find that news organizations express their ideological bias not by directly advocating for a preferred political party, but rather by disproportionately criticizing one side, a con…
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In contrast to our multi-dimensional approach that examines the ideology of the issue being discussed instead of the author stance, much of the recent work in computational linguistics has been dedicated to the latter while collapsing ideology to one dimension.
“In contrast to our multi-dimensional approach that examines the ideology of the issue being discussed instead of the author stance, much of the recent work in computational linguistics has been dedicated to the latter (detection of ideological bias in news media) while collapsing ideology to one dimension”
From Political Ideology and Polarization: A Multi-dimensional Approach (2022) · cites this paper as Budak et al., 2016
Cites
Cortes, C., & Vapnik, V. (1995) · Machine Learning · doi:10.1007/bf00994018 · Cited by 59,421 · see citing works
(2003) · Journal of Machine Learning Research · doi:10.5555/944919.944937 · Cited by 32,337 · see citing works
Amazon's Mechanical Turk: A New Source of Inexpensive, Yet High-Quality, Data?
Buhrmester, M., Kwang, T., & Gosling, S. D. (2011) · Perspectives on Psychological Science · doi:10.1177/1745691610393980 · Cited by 9,996 · see citing works
The Agenda-Setting Function of Mass Media
McCombs, M. E., & Shaw, D. L. (1972) · Public Opinion Quarterly · doi:10.1086/267990 · Cited by 9,537 · see citing works
Large-Scale Machine Learning with Stochastic Gradient Descent
Bottou, L. (2010) · Proceedings of COMPSTAT'2010 · doi:10.1007/978-3-7908-2604-3_16 · Cited by 5,795 · see citing works
Media Discourse and Public Opinion on Nuclear Power: A Constructionist Approach
Gamson, W. A., & Modigliani, A. (1989) · American Journal of Sociology · doi:10.1086/229213 · Cited by 4,997 · see citing works
(2009) · Minds at UW (University of Wisconsin) · Cited by 4,791 · see citing works
A generalization of sampling without replacement from a finite universe
Horvitz, D. G., & Thompson, D. J. (1952) · Journal of the American Statistical Association · doi:10.1080/01621459.1952.10483446 · Cited by 4,293 · see citing works
Is anyone responsible? How television frames political issues.
Iyengar, S. (1991) · University of Chicago Press eBooks · doi:10.7208/chicago/9780226388533.001.0001 · Cited by 4,258 · see citing works
Evaluating Online Labor Markets for Experimental Research: Amazon.com's Mechanical Turk
Berinsky, A. J., Huber, G. A., & Lenz, G. S. (2012) · Political Analysis · doi:10.1093/pan/mpr057 · Cited by 4,008 · see citing works
Running experiments on Amazon Mechanical Turk
Paolacci, G., Chandler, J., & Ipeirotis, P. G. (2010) · Judgment and Decision Making · doi:10.1017/s1930297500002205 · Cited by 3,767 · see citing works
Exposure to ideologically diverse news and opinion on Facebook
Bakshy, E., Messing, S., & Adamic, L. A. (2015) · Science · doi:10.1126/science.aaa1160 · Cited by 3,245 · see citing works
Conducting behavioral research on Amazon’s Mechanical Turk
Mason, W., & Suri, S. (2012) · Behavior Research Methods · doi:10.3758/s13428-011-0124-6 · Cited by 3,172 · see citing works
(1987) · Cited by 2,656 · see citing works
Data collection in a flat world: the strengths and weaknesses of Mechanical Turk samples
Goodman, J. K., Cryder, C. E., & Cheema, A. (2013) · Journal of Behavioral Decision Making · doi:10.1002/bdm.1753 · Cited by 2,242 · see citing works
Filter Bubbles, Echo Chambers, and Online News Consumption
Flaxman, S., Goel, S., & Rao, J. M. (2016) · Public Opinion Quarterly · doi:10.1093/poq/nfw006 · Cited by 1,992 · see citing works
Self-citationCrowdsourcing user studies with Mechanical Turk
Kittur, A., Ed, H., & Suh, B. (2008) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/1357054.1357127 · Cited by 1,941 · see citing works
Red Media, Blue Media: Evidence of Ideological Selectivity in Media Use
Iyengar, S., & Hahn, K. S. (2009) · Journal of Communication · doi:10.1111/j.1460-2466.2008.01402.x · Cited by 1,940 · see citing works
Questions and answers in attitude surveys: Experiments on question form, wording, and context
(1990) · Computers Environment and Urban Systems · doi:10.1016/0198-9715(90)90054-w · Cited by 1,718 · see citing works
What Drives Media Slant? Evidence From U.S. Daily Newspapers
Gentzkow, M. A., & Shapiro, J. M. (2010) · Econometrica · doi:10.3982/ecta7195 · Cited by 1,559 · see citing works
Media Framing of a Civil Liberties Conflict and Its Effect on Tolerance
Nelson, T. E., Clawson, R. A., & Oxley, Z. M. (1997) · American Political Science Review · doi:10.2307/2952075 · Cited by 1,422 · see citing works
Patterson, T. E. (1993) · Medical Entomology and Zoology · Cited by 1,306 · see citing works
Gentzkow, M., & Shapiro, J. M. (2006) · Journal of Political Economy · doi:10.1086/499414 · Cited by 1,274 · see citing works
Mullainathan, S., & Shleifer, A. (2005) · American Economic Review · doi:10.1257/0002828054825619 · Cited by 1,236 · see citing works
Toward a Psychology of Framing Effects
Nelson, T. E., Oxley, Z., & Clawson, R. A. (1997) · Political Behavior · doi:10.1023/a:1024834831093 · Cited by 1,028 · see citing works
Groseclose, T., & Milyo, J. D. (2005) · The Quarterly Journal of Economics · doi:10.1162/003355305775097542 · Cited by 1,018 · see citing works
Ideological Segregation Online and Offline *
Gentzkow, M., & Shapiro, J. M. (2011) · The Quarterly Journal of Economics · doi:10.1093/qje/qjr044 · Cited by 980 · see citing works
Vallone, R. P., Ross, L. D., & Lepper, M. R. (1985) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.49.3.577 · Cited by 894 · see citing works
Issue Frames and Group-Centrism in American Public Opinion
Nelson, T. E., & Kinder, D. R. (1996) · The Journal of Politics · doi:10.2307/2960149 · Cited by 799 · see citing works
Altering the Foundations of Support for the President Through Priming
Krosnick, J. A., & Kinder, D. R. (1990) · American Political Science Review · doi:10.2307/1963531 · Cited by 794 · see citing works
Questions and Answers in Attitude Surveys.
Robinson, J. P., Schuman, H., & Presser, S. (1983) · Social Forces · doi:10.2307/2578327 · Cited by 781 · see citing works
Gentzkow, M. A., & Shapiro, J. M. (2005) · National Bureau of Economic Research · doi:10.3386/w11664 · Cited by 631 · see citing works
Utility data annotation with Amazon Mechanical Turk
Sorokin, A. A., & Forsyth, D. A. (2008) · 2009 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops · doi:10.1109/cvprw.2008.4562953 · Cited by 620 · see citing works
What Drives Media Slant? Evidence from U.S. Daily Newspapers
Gentzkow, M. A., & Shapiro, J. M. (2006) · National Bureau of Economic Research · doi:10.3386/w12707 · Cited by 579 · see citing works
New Media and the Polarization of American Political Discourse
Baum, M., & Groeling, T. J. (2008) · Political Communication · doi:10.1080/10584600802426965 · Cited by 549 · see citing works
Amazon's Mechanical Turk: A new source of inexpensive, yet high-quality, data?
Buhrmester, M. D., Kwang, T., & Gosling, S. D. (2011) · PsycEXTRA Dataset · doi:10.1037/e527772014-223 · Cited by 534 · see citing works
Media bias in presidential elections: a meta-analysis
D'Alessio, D., & Allen, M. (2000) · Journal of Communication · doi:10.1111/j.1460-2466.2000.tb02866.x · Cited by 510 · see citing works
The Political Culture of Social Welfare Policy
Gamson, W. A., & Lasch, K. (1983) · Elsevier eBooks · doi:10.1016/b978-0-12-657980-2.50032-2 · Cited by 467 · see citing works
Fast, cheap, and creative: evaluating translation quality using Amazon's Mechanical Turk
Callison-Burch, C. (2009) · Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing Volume 1 - EMNLP '09 · doi:10.3115/1699510.1699548 · Cited by 412 · see citing works
Iyengar, S., & Kinder, D. R. (2010) · University of Chicago Press eBooks · doi:10.7208/chicago/9780226388601.001.0001 · Cited by 325 · see citing works
Ego-Involvement and the Third Person Effect of Televised News Coverage
Perloff, R. M. (1989) · Communication Research · doi:10.1177/009365089016002004 · Cited by 303 · see citing works
The Impact of Individual and Interpersonal Factors on Perceived News Media Bias
Eveland, W. P., & Shah, D. V. (2003) · Political Psychology · doi:10.1111/0162-895x.00318 · Cited by 277 · see citing works
Newspaper Coverage of Political Scandals
Puglisi, R., & Snyder, J. M. (2011) · The Journal of Politics · doi:10.1017/s0022381611000569 · Cited by 271 · see citing works
News Production: Theory and Practice
Niblock, S., & Machín, D. (2006) · Cited by 165 · see citing works
Measuring Explicit Political Positions of Media
Ho, D. E., & Quinn, K. M. (2008) · Quarterly Journal of Political Science · doi:10.1561/100.00008048 · Cited by 145 · see citing works
Linguistically debatable or just plain wrong?
Plank, B., Hovy, D., & Søgaard, A. (2014) · Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) · doi:10.3115/v1/p14-2083 · Cited by 86 · see citing works
Classifying the Political Leaning of News Articles and Users from User Votes
Zhou, D. X., Resnick, P., & Mei, Q. (2021) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v5i1.14108 · Cited by 84 · see citing works
Amazon Mechanical Turk for Subjectivity Word Sense Disambiguation
Akkaya, C., Conrad, A. P., Wiebe, J., & Mihalcea, R. F. (2010) · University of North Texas Digital Library (University of North Texas) · Cited by 68 · see citing works
Selectively Recruiting Participants from Amazon Mechanical Turk Using QualtricsPreprint
Péer, E., Paolacci, G., Chandler, J., & Mueller, P. A. (2012) · SSRN Electronic Journal · doi:10.2139/ssrn.2100631 · Cited by 67 · see citing works
News That Matters: Television and American Opinion, Updated Edition
Iyengar, S., & Kinder, D. R. (1987) · Cited by 43 · see citing works
TOWARD A PSYCHOLOGY OF FRAMING
Nelson, T. E., Oxley, Z. M., & Clawson, R. A. (1997) · Cited by 11 · see citing works
Flagged first, then by citations
1 of the 52 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
Exposure to ideologically diverse news and opinion on Facebook
Bakshy, E., Messing, S., & Adamic, L. A. (2015) · Science · doi:10.1126/science.aaa1160 · Cited by 3,245 · see citing works
Fake news on Twitter during the 2016 U.S. presidential election
Grinberg, N., Joseph, K., Friedland, L., Swire-Thompson, B., & Lazer, D. (2019) · Science · doi:10.1126/science.aau2706 · Cited by 1,803 · see citing works
Frenemies: How Social Media Polarizes America
Settle, J. E. (2018) · doi:10.1017/9781108560573 · Cited by 377 · see citing works
(Almost) Everything in Moderation: New Evidence on Americans' Online Media Diets
Guess, A. M. (2021) · American Journal of Political Science · doi:10.1111/ajps.12589 · Cited by 342 · see citing works
Asymmetric ideological segregation in exposure to political news on Facebook
González-Bailón, S., Lazer, D., Barberá, P., Zhang, M., Allcott, H., Brown, T., Crespo-Tenorio, A., Freelon, D., Gentzkow, M., Guess, A. M., et al. (2023) · Science · doi:10.1126/science.ade7138 · Cited by 315 · see citing works
Metzger, M. J., Hartsell, E. H., & Flanagin, A. J. (2020) · Communication Research · doi:10.1177/0093650215613136 · Cited by 272 · see citing works
Hasell, A., & Weeks, B. E. (2016) · Human Communication Research · doi:10.1111/hcre.12092 · Cited by 246 · see citing works
Auditing Partisan Audience Bias within Google Search
Robertson, R. E., Jiang, S., Joseph, K., Friedland, L., Lazer, D., & Wilson, C. (2018) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3274417 · Cited by 209 · see citing works
Conservatives’ susceptibility to political misperceptions
Garrett, R. K., & Bond, R. M. (2021) · Science Advances · doi:10.1126/sciadv.abf1234 · Cited by 176 · see citing works
Stier, S., Kirkizh, N., Froio, C., & Schroeder, R. (2020) · The International Journal of Press/Politics · doi:10.1177/1940161220907018 · Cited by 171 · see citing works
Partisan selective exposure in online news consumption: evidence from the 2016 presidential campaign
Peterson, E., Goel, S., & Iyengar, S. (2019) · Political Science Research and Methods · doi:10.1017/psrm.2019.55 · Cited by 156 · see citing works
Self-citationLi, C., & Goldwasser, D. (2019) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p19-1247 · Cited by 147 · see citing works
Using the president’s tweets to understand political diversion in the age of social media
Lewandowsky, S., Jetter, M., & Ecker, U. K. H. (2020) · Nature Communications · doi:10.1038/s41467-020-19644-6 · Cited by 139 · see citing works
We Can Detect Your Bias: Predicting the Political Ideology of News Articles
Baly, R., Martino, G. D. S., Glass, J. R., & Nakov, P. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.404 · Cited by 142 · see citing works
Defining the Enemy: How Donald Trump Frames the News Media
Meeks, L. M. (2019) · Journalism & Mass Communication Quarterly · doi:10.1177/1077699019857676 · Cited by 130 · see citing works
What to Believe? Social Media Commentary and Belief in Misinformation
Anspach, N. M., & Carlson, T. N. (2018) · Political Behavior · doi:10.1007/s11109-018-9515-z · Cited by 140 · see citing works
Inside the Right-Leaning Echo Chambers: Characterizing Gab, an Unmoderated Social System
Lima, L., Reis, J. C. S., Melo, P. D. F., Murai, F., Araújo, L., Vikatos, P., & Benevenuto, F. (2018) · IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM) · doi:10.1109/asonam.2018.8508809 · Cited by 110 · see citing works
In-Group Love and Out-Group Hate: White Racial Attitudes in Contemporary U.S. Elections
Jardina, A. E. (2020) · Political Behavior · doi:10.1007/s11109-020-09600-x · Cited by 105 · see citing works
Traberg, C. S., & van der Linden, S. (2021) · Personality and Individual Differences · doi:10.1016/j.paid.2021.111269 · Cited by 104 · see citing works
Schmidtke, H. (2018) · The Review of International Organizations · doi:10.1007/s11558-018-9320-9 · Cited by 103 · see citing works
Search bias quantification: investigating political bias in social media and web search
Kulshrestha, J., Eslami, M., Messias, J., Zafar, M. B., Ghosh, S., Gummadi, K. P., & Karahalios, K. G. (2018) · Information Retrieval · doi:10.1007/s10791-018-9341-2 · Cited by 103 · see citing works
Shared Emotion: The Social Amplification of Partisan News on Twitter
Hasell, A. (2020) · Digital Journalism · doi:10.1080/21670811.2020.1831937 · Cited by 97 · see citing works
Exposure to news grows less fragmented with an increase in mobile access
Yang, T., Majó-Vázquez, S., Nielsen, R. K., & González-Bailón, S. (2020) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2006089117 · Cited by 95 · see citing works
Through the Grapevine: Informational Consequences of Interpersonal Political Communication
CARLSON, T. N. (2019) · American Political Science Review · doi:10.1017/s000305541900008x · Cited by 95 · see citing works
Rodrigo-Ginés, F.-J., Carrillo‐de‐Albornoz, J., & Plaza, L. (2023) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.121641 · Cited by 95 · see citing works
Polarized information ecosystems can reorganize social networks via information cascades
Tokita, C. K., Guess, A. M., & Tarnita, C. E. (2021) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2102147118 · Cited by 92 · see citing works
Peng, Y. (2018) · Journal of Communication · doi:10.1093/joc/jqy041 · Cited by 92 · see citing works
In Plain Sight: Media Bias Through the Lens of Factual Reporting
Fan, L., White, M., Sharma, E., Su, R., Choubey, P. K., Huang, R., & Wang, L. (2019) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d19-1664 · Cited by 92 · see citing works
The Structure of Toxic Conversations on Twitter
Saveski, M., Roy, B., & Roy, D. K. (2021) · International World Wide Web Conference (WWW) · doi:10.1145/3442381.3449861 · Cited by 88 · see citing works
The Web of False Information: Rumors, Fake News, Hoaxes, Clickbait, and Various Other Shenanigans
Zannettou, S., Sirivianos, M., Blackburn, J., & Kourtellis, N. (2019) · Journal of Data and Information Quality · doi:10.1145/3309699 · Cited by 86 · see citing works
There is no liberal media bias in which news stories political journalists choose to cover
Hassell, H. J. G., Holbein, J. B., & Miles, M. R. (2020) · Science Advances · doi:10.1126/sciadv.aay9344 · Cited by 86 · see citing works
Aligning AI Optimization to Community Well-Being
Stray, J. (2020) · International Journal of Community Well-Being · doi:10.1007/s42413-020-00086-3 · Cited by 78 · see citing works
Breaking the News: First Impressions Matter on Online News
Rieis, J., Souza, F., de Melo, P. O. S. V., Prates, R. O., Kwak, H., & An, J. (2021) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v9i1.14619 · Cited by 71 · see citing works
Mozer, R., Miratrix, L., Kaufman, A., & Anastasopoulos, L. (2018) · Political Analysis · doi:10.1017/pan.2020.1 · Cited by 73 · see citing works
What happened? The Spread of Fake News Publisher Content During the 2016 U.S. Presidential Election
Budak, C. (2019) · International World Wide Web Conference (WWW) · doi:10.1145/3308558.3313721 · Cited by 65 · see citing works
Self-citationHigh level of correspondence across different news domain quality rating sets
Lin, H., Lasser, J., Lewandowsky, S., Cole, R., Gully, A., Rand, D. G., & Pennycook, G. (2023) · PNAS Nexus · doi:10.1093/pnasnexus/pgad286 · Cited by 75 · see citing works
Media Competition and Social Disagreement
Perego, J., & Yuksel, S. (2022) · Econometrica · doi:10.3982/ecta16417 · Cited by 37 · see citing works
Are there still limits on partisan prejudice?
Westwood, S. J., Peterson, E., & Lelkes, Y. (2019) · Public Opinion Quarterly · doi:10.1093/poq/nfz034 · Cited by 61 · see citing works
The Web of False Information: Rumors, Fake News, Hoaxes, Clickbait, and Various Other ShenanigansPreprint
Zannettou, S., Sirivianos, M., Blackburn, J., & Kourtellis, N. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1804.03461 · Cited by 60 · see citing works
What Was Written vs. Who Read It: News Media Profiling Using Text Analysis and Social Media Context
Baly, R., Karadzhov, G., An, J., Kwak, H., Dinkov, Y., Ali, A. M. A., Glass, J. R., & Nakov, P. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.308 · Cited by 64 · see citing works
Detecting Political Bias in News Articles Using Headline Attention
Gangula, R. R. R., Duggenpudi, S. R., & Mamidi, R. (2019) · BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP (BlackboxNLP) · doi:10.18653/v1/w19-4809 · Cited by 58 · see citing works
Robust Fake News Detection Over Time and Attack
Horne, B. D., Nørregaard, J., & Adalı, S. (2019) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3363818 · Cited by 59 · see citing works
Content Analysis by the Crowd: Assessing the Usability of Crowdsourcing for Coding Latent Constructs
Lind, F., Gruber, M., & Boomgaarden, H. G. (2017) · Communication Methods and Measures · doi:10.1080/19312458.2017.1317338 · Cited by 57 · see citing works
Frenemies: How Social Media Polarizes America
Settle, J. E. (2018) · Cited by 56 · see citing works
More Accounts, Fewer Links: How Algorithmic Curation Impacts Media Exposure in Twitter Timelines
Bandy, J., & Diakopoulos, N. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3449152 · Cited by 54 · see citing works
On Measuring Bias in Online Information
Pitoura, E., Tsaparas, P., Flouris, G., Fundulaki, I., Papadakos, P., Abiteboul, S., & Weikum, G. (2018) · ACM SIGMOD Record · doi:10.1145/3186549.3186553 · Cited by 52 · see citing works
Auditing the Partisanship of Google Search Snippets
Hu, D., Jiang, S., Robertson, R. E., & Wilson, C. (2019) · International World Wide Web Conference (WWW) · doi:10.1145/3308558.3313654 · Cited by 51 · see citing works
Accuracy and Political Bias of News Source Credibility Ratings by Large Language Models
Yang, K.-C., & Menczer, F. (2025) · Proceedings of the 17th ACM Web Science Conference 2025 · doi:10.1145/3717867.3717903 · Cited by 81 · see citing works
(2017) · International journal of communication · Cited by 49 · see citing works
Peterson, E., & Kagalwala, A. (2021) · American Political Science Review · doi:10.1017/s0003055420001124 · Cited by 49 · see citing works
Wojcieszak, M., Winter, S., & Yu, X. (2020) · Mass Communication & Society · doi:10.1080/15205436.2020.1714663 · Cited by 47 · see citing works
The advantage of the right in social media news sharing
González‐Bailón, S., d’Andrea, V., Freelon, D., & Domenico, M. D. (2022) · PNAS Nexus · doi:10.1093/pnasnexus/pgac137 · Cited by 46 · see citing works
Evaluation metrics for measuring bias in search engine results
Gezi̇ci, G., Lipani, A., Saygın, Y., & Yılmaz, E. (2021) · Information Retrieval · doi:10.1007/s10791-020-09386-w · Cited by 45 · see citing works
Bilateral Tensions, the Trade War, and US–China Trade Relations
Zeng, K., Wells, R., Gu, J., & Wilkins, A. (2022) · Business and Politics · doi:10.1017/bap.2022.8 · Cited by 44 · see citing works
Characterizing Information Diets of Social Media Users
Kulshrestha, J., Zafar, M. B., Espín-Noboa, L., Gummadi, K. P., & Ghosh, S. (2021) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v9i1.14634 · Cited by 44 · see citing works
Climate Change Marches as Motivators for Bystander Collective Action
Swim, J. K., Geiger, N., & Lengieza, M. L. (2019) · Frontiers in Communication · doi:10.3389/fcomm.2019.00004 · Cited by 44 · see citing works
Rating Reliability and Bias in News Articles: Does AI Assistance Help Everyone?
Horne, B. D., Nevo, D., O’Donovan, J., Cho, J.-H., & Adalı, S. (2019) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v13i01.3226 · Cited by 49 · see citing works
Partisan media, untrustworthy news sites, and political misperceptions
Weeks, B. E., Menchen-Trevino, E., Calabrese, C. R., Casas, A., & Wojcieszak, M. (2021) · New Media & Society · doi:10.1177/14614448211033300 · Cited by 43 · see citing works
Does the Ideology of the Newsroom Affect the Provision of Media Slant?
Hassell, H. J. G., Miles, M. R., & Reuning, K. (2021) · Political Communication · doi:10.1080/10584609.2021.1986613 · Cited by 42 · see citing works
Political Discussion is Abundant in Non-political Subreddits (and Less Toxic)
Rajadesingan, A., Budak, C., & Resnick, P. (2021) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v15i1.18081 · Cited by 41 · see citing works
Self-citationThe mass, fake news, and cognition security
Guo, B., Ding, Y., Sun, Y., Ma, S., Li, K., & Yu, Z. (2020) · Frontiers of Computer Science · doi:10.1007/s11704-020-9256-0 · Cited by 37 · see citing works
Conservative news nonprofits: Claiming legitimacy without transparency
Buozis, M., & Konieczna, M. (2021) · Journalism · doi:10.1177/14648849211056145 · Cited by 35 · see citing works
Sentiment dynamics in social media news channels
Kumar, N., Nagalla, R., Marwah, T., & Singh, M. (2018) · Online Social Networks and Media · doi:10.1016/j.osnem.2018.10.004 · Cited by 35 · see citing works
Németh, R. (2022) · Journal of Computational Social Science · doi:10.1007/s42001-022-00196-2 · Cited by 34 · see citing works
Ideology and Composition Among an Online Crowd: Evidence from Wikipedians
Greenstein, S. M., Gu, G., & Zhu, F. (2020) · Management Science · doi:10.1287/mnsc.2020.3661 · Cited by 32 · see citing works
Koltsova, O. Y., & Пашахин, С. (2019) · Media War & Conflict · doi:10.1177/1750635219829876 · Cited by 29 · see citing works
Waddell, T. F. (2019) · Journalism Practice · doi:10.1080/17512786.2019.1585197 · Cited by 30 · see citing works
Tien, J. H., Eisenberg, M. C., Cherng, S. T., & Porter, M. A. (2020) · Applied Network Science · doi:10.1007/s41109-019-0223-3 · Cited by 29 · see citing works
A Multidimensional Dataset Based on Crowdsourcing for Analyzing and Detecting News Bias
Färber, M., Burkard, V., Jatowt, A., & Lim, S. (2020) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3340531.3412876 · Cited by 28 · see citing works
Yu, H., Yang, C., Yu, P., & Liu, K. (2022) · PLoS ONE · doi:10.1371/journal.pone.0264794 · Cited by 28 · see citing works
A machine learning approach to open public comments for policymaking
Ingrams, A. R. (2020) · Information Polity · doi:10.3233/ip-200256 · Cited by 28 · see citing works
Datta, P. M., Whitmore, M., & Nwankpa, J. K. (2021) · Digital Threats Research and Practice · doi:10.1145/3428157 · Cited by 32 · see citing works
From human invaders to problem bears: A media content analysis of grizzly bear conservation
Hughes, C., Foote, A. L., Yarmey, N. T., Hwang, C., Thorlakson, J. J. D., & Nielsen, S. E. (2020) · Conservation Science and Practice · doi:10.1111/csp2.176 · Cited by 26 · see citing works
News Sharing on Social Media: Mapping the Ideology of News Media, Politicians, and the Mass Public
Eady, G., Bonneau, R., Tucker, J. A., & Nagler, J. (2024) · Political Analysis · doi:10.1017/pan.2024.19 · Cited by 26 · see citing works
Reiss, M. V. (2022) · Digital Journalism · doi:10.1080/21670811.2022.2105243 · Cited by 37 · see citing works
Liu, J., Siegel, L. N., Gibson, L., Kim, Y., Binns, S., Emery, S., & Hornik, R. (2019) · Journal of Communication · doi:10.1093/joc/jqz033 · Cited by 28 · see citing works
Li, C., & Goldwasser, D. (2021) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2021.findings-acl.401 · Cited by 23 · see citing works
Identifying Political Bias in News Articles.
Lazaridou, K., & Krestel, R. (2016) · Bull. IEEE Tech. Comm. Digit. Libr. · Cited by 24 · see citing works
Papakyriakopoulos, O., & Zuckerman, E. (2021) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v15i1.18076 · Cited by 23 · see citing works
Campbell, E., Kotcher, J., Maibach, E. W., Rosenthal, S. A., & Leiserowitz, A. (2021) · Current Research in Ecological and Social Psychology · doi:10.1016/j.cresp.2021.100008 · Cited by 22 · see citing works
Message Impartiality in Social Media Discussions
Zafar, M. B., Gummadi, K. P., & Danescu-Niculescu-Mizil, C. (2021) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v10i1.14754 · Cited by 22 · see citing works
Coding latent concepts: a human and LLM-coordinated content analysis procedure
Jia, F., Ai, Y., Liu, X., Deng, Y., & Li, Y. (2024) · Communication Research Reports · doi:10.1080/08824096.2024.2410263 · Cited by 22 · see citing works
The Perils of Classifying Political Orientation From Text.
Hao, Y., Lavoie, A., & Das, S. (2017) · International Joint Conference on Artificial Intelligence · Cited by 21 · see citing works
Estimating Ideal Points of Newspapers from Editorial Texts
Kaneko, T., Asano, T.-A., & Miwa, H. (2020) · The International Journal of Press/Politics · doi:10.1177/1940161220935058 · Cited by 21 · see citing works
Ideological Segregation among Online Collaborators: Evidence from Wikipedians
Greenstein, S. M., Gu, Y. X., & Zhu, F. (2016) · National Bureau of Economic Research · doi:10.3386/w22744 · Cited by 21 · see citing works
Empirical Evaluation of Three Common Assumptions in Building Political Media Bias Datasets
Ganguly, S., Kulshrestha, J., An, J., & Kwak, H. (2020) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v14i1.7362 · Cited by 20 · see citing works
Wojcieszak, M., Sobkowicz, P., Yu, X., & Bulat, B. (2021) · The International Journal of Press/Politics · doi:10.1177/19401612211004418 · Cited by 24 · see citing works
High level of correspondence across different news domain quality rating setsPreprint
Lin, H., Lasser, J., Lewandowsky, S., Cole, R., Gully, A., Rand, D. G., & Pennycook, G. (2022) · doi:10.31234/osf.io/qy94s · Cited by 20 · see citing works
Jahani, E., Gallagher, N. M., Merhout, F., Cavalli, N., Guilbeault, D., Leng, Y., & Bail, C. A. (2022) · Scientific Reports · doi:10.1038/s41598-022-23673-0 · Cited by 19 · see citing works
Ideological self-selection in online news exposure: Evidence from Europe and the US
Mangold, F., Schoch, D., & Stier, S. (2024) · Science Advances · doi:10.1126/sciadv.adg9287 · Cited by 21 · see citing works
Hwang, J., Dahir, N., Sarukkai, M., & Wright, G. (2023) · Sociological Methods & Research · doi:10.1177/00491241231171945 · Cited by 17 · see citing works
Partisan Bias in Economic News Content: New Evidence
Merkley, E. (2018) · American Politics Research · doi:10.1177/1532673x18821954 · Cited by 17 · see citing works
Measuring Partisan Media Bias Cross‐Nationally
Castro, L. (2021) · Swiss Political Science Review · doi:10.1111/spsr.12459 · Cited by 16 · see citing works
Revealing Media Bias in News Articles
Hamborg, F. (2023) · Springer eBooks · doi:10.1007/978-3-031-17693-7 · Cited by 18 · see citing works
Supervised Learning for Misinformation Detection in WhatsApp
Reis, J. C. S., & Benevenuto, F. (2021) · Brazilian Symposium on Multimedia and the Web · doi:10.1145/3470482.3479641 · Cited by 16 · see citing works
Analysis of Media Bias in Policy Discourse in India
Sen, A., Ghatak, D., Khanuja, G., Rekha, K., Gupta, M., Dhakate, S., Sharma, K., & Seth, A. (2022) · ACM SIGCAS/SIGCHI Conference on Computing and Sustainable Societies (COMPASS) · doi:10.1145/3530190.3534798 · Cited by 16 · see citing works
From Viewers to Voters: Tracing Fox News' Impact on American DemocracyPreprint
Ash, E., Galletta, S., Pinna, M., & Warshaw, C. (2021) · SSRN Electronic Journal · doi:10.2139/ssrn.3837457 · Cited by 16 · see citing works
Reducing Cross-Topic Political Homogenization in Content-Based News Recommendation
Shivaram, K., Liu, P., Shapiro, M. A., Bilgic, M., & Culotta, A. (2022) · ACM Conference on Recommender Systems (RecSys) · doi:10.1145/3523227.3546782 · Cited by 15 · see citing works
Identifying Media Bias by Analyzing Reported Speech
Lazaridou, K., Krestel, R., & Naumann, F. (2017) · Proceedings · doi:10.1109/icdm.2017.119 · Cited by 15 · see citing works
- Authors
- Budak, C., Goel, S., & Rao, J. M.
- Affiliations
- Microsoft (United States) (US)
- Corresponding author
- Budak, Ceren
- Venue
- Public Opinion Quarterly
- Publisher
- Oxford University Press (OUP)
- Published
- 2016
- Volume
- 80
- Issue
- S1
- Pages
- 250-271
- ISSN
- 0033-362X (ISSN-L)
- 1537-5331
- OpenAlex
- W1119807432
- Cited by
- 336
- References
- 52
- Citations by year
- 2026: 18 · 2025: 40 · 2024: 28 · 2023: 38 · 2022: 30 · 2021: 55 · 2020: 38 · 2019: 39 · 2018: 32 · 2017: 9 · 2016: 5 · 2015: 3 · 2014: 1
- FWCI
- 81.00
- Type
- journal article
- Language
- en
- Open access
- green
- doi.org
- Locations
- Public Opinion Quarterly (publishedVersion)
- SSRN Electronic Journal (acceptedVersion)
- HighWire Press Open Archive (submittedVersion)
- Topics
- Social Sciences › Sociology and Political Science › Media Influence and Politics
- Social Sciences › Communication › Media Studies and Communication
- Social Sciences › General Social Sciences › Computational and Text Analysis Methods
- Keywords
- media bias, ideological bias, news framing, media coverage of politics, partisan bias, political scandal, news selection
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
