Lower Quantity, Higher Quality: Auditing News Content and User Perceptions on Twitter/X Algorithmic versus Chronological Timelines
Open accessAbstract
Social media personalization algorithms increasingly influence the flow of civic information through society, resulting in concerns about "filter bubbles'', "echo chambers'', and other ways they might exacerbate ideological segregation and fan the spread of polarizing content. To address these concerns, we designed and conducted a sociotechnical audit (STA) to investigate how Twitter/X's timeline algorithm affects news curation while also tracking how user perceptions change in response. We deployed a custom-built system that, over the course of three weeks, passively tracked all tweets loaded in users' browsers in the first week, then in the second week enacted an intervention to users' Twitter/X homepage to restrict their view to only the algorithmic or chronological timeline (randomized). We flipped this condition for each user in the third week. We ran our audit in late 2023, collecting user-centered metrics (self-reported survey measures) and platform-centered metrics (views, clicks, likes) for 243 users, along with over 800,000 tweets. Using the STA framework, our results are two-fold: (1) Our algorithm audit finds that Twitter/X's algorithmic timeline resulted in a lower quantity but higher quality of news --- less ideologically congruent, less extreme, and slightly more reliable --- compared to the chronological timeline. (2) Our user audit suggests that although our timeline intervention had significant effects on users' behaviors, it had little impact on their overall perceptions of the platform. Our paper discusses these findings and their broader implications in the context of algorithmic news curation, user-centric audits, and avenues for independent social science research.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Earlier research suggests that personalized recommendations tend to be more centrist in political stance.
“However, our findings diverge from earlier research suggesting that personalized recommendations tend to be more centrist in political stance [8,9,29].”
From Auditing Political Exposure Bias: Algorithmic Amplification on Twitter/X During the 2024 U.S. Presidential Election (2025) · cites this paper as [29]
Other studies report increased exposure to ideologically aligned friends but decreased exposure to external links.
“Other studies report increased exposure to ideologically aligned friends [4,8], but decreased exposure to external links [4,29].”
From Auditing Political Exposure Bias: Algorithmic Amplification on Twitter/X During the 2024 U.S. Presidential Election (2025) · cites this paper as [29]
Bakshy et al. report considerable cross-cutting exposure on Facebook, and Wang et al. find that Twitter/X provides higher-quality and less ideologically congruent news curation.
“Bakshy et al. [2] report considerable cross-cutting exposure on Facebook, and Wang et al. [29] find that Twitter/X provides higher-quality and less ideologically congruent news curation.”
From Auditing Political Exposure Bias: Algorithmic Amplification on Twitter/X During the 2024 U.S. Presidential Election (2025) · cites this paper as [29]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Colman (2022) · doi:10.1073/pnas · Cited by 4,370 · see citing works
Lazer, D. M. J., Baum, M. A., Benkler, Y., Berinsky, A. J., Greenhill, K. M., Menczer, F., Metzger, M. J., Nyhan, B., Pennycook, G., Rothschild, D., et al. (2018) · Science · doi:10.1126/science.aao2998 · Cited by 4,160 · see citing works
Exposure to ideologically diverse news and opinion on Facebook
Bakshy, E., Messing, S., & Adamic, L. A. (2015) · Science · doi:10.1126/science.aaa1160 · Cited by 3,245 · see citing works
The echo chamber effect on social media
Cinelli, M., De Francisci Morales, G., Galeazzi, A., Quattrociocchi, W., & Starnini, M. (2021) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2023301118 · Cited by 2,092 · cited 1× · see citing works
“Many studies have investigated YouTube video recommendations in response to such concerns, finding consistent radicalization results for a small proportion of politically extreme users [23, 31, 48].”
Filter Bubbles, Echo Chambers, and Online News Consumption
Flaxman, S., Goel, S., & Rao, J. M. (2016) · Public Opinion Quarterly · doi:10.1093/poq/nfw006 · Cited by 1,992 · see citing works
The filter bubble: how the new personalized web is changing what we read and how we think
(2012) · Choice Reviews Online · doi:10.5860/choice.50-0926 · Cited by 1,319 · see citing works
Advertising Content and Consumer Engagement on Social Media: Evidence from Facebook
Lee, D., Hosanagar, K., & Nair, H. S. (2018) · Management Science · doi:10.1287/mnsc.2017.2902 · Cited by 1,069 · see citing works
Ideological Segregation Online and Offline *
Gentzkow, M., & Shapiro, J. M. (2011) · The Quarterly Journal of Economics · doi:10.1093/qje/qjr044 · Cited by 980 · see citing works
Messing, S., & Westwood, S. J. (2014) · Communication Research · doi:10.1177/0093650212466406 · Cited by 902 · see citing works
Personalized news recommendation based on click behavior
Liu, J., Dolan, P., & Pedersen, E. R. (2010) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/1719970.1719976 · Cited by 772 · see citing works
Curated Flows: A Framework for Mapping Media Exposure in the Digital Age
Thorson, K., & Wells, C. (2015) · Communication Theory · doi:10.1111/comt.12087 · Cited by 764 · see citing works
"I always assumed that I wasn't really that close to [her]"
Eslami, M., Rickman, A., Vaccaro, K., Aleyasen, A., Vuong, A., Karahalios, K. G., Hamilton, K., & Sandvig, C. (2015) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/2702123.2702556 · Cited by 665 · see citing works
Social Cognition and Social Perception
Fiske, S. T. (1993) · Annual Review of Psychology · doi:10.1146/annurev.ps.44.020193.001103 · Cited by 613 · see citing works
How do social media feed algorithms affect attitudes and behavior in an election campaign?
Guess, A. M., Malhotra, N., Pan, J., Barberá, P., Allcott, H., Brown, T., Crespo-Tenorio, A., Dimmery, D., Freelon, D., Gentzkow, M., et al. (2023) · Science · doi:10.1126/science.abp9364 · Cited by 363 · see citing works
(Almost) Everything in Moderation: New Evidence on Americans' Online Media Diets
Guess, A. M. (2021) · American Journal of Political Science · doi:10.1111/ajps.12589 · Cited by 342 · see citing works
Computational Research in the Post-API Age
Freelon, D. (2018) · Political Communication · doi:10.1080/10584609.2018.1477506 · Cited by 336 · see citing works
Asymmetric ideological segregation in exposure to political news on Facebook
González-Bailón, S., Lazer, D., Barberá, P., Zhang, M., Allcott, H., Brown, T., Crespo-Tenorio, A., Freelon, D., Gentzkow, M., Guess, A. M., et al. (2023) · Science · doi:10.1126/science.ade7138 · Cited by 315 · see citing works
DeVito, M. A., Gergle, D., & Birnholtz, J. (2017) · Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3025453.3025659 · Cited by 311 · see citing works
From ranking algorithms to ‘ranking cultures’
Rieder, B., Matamoros-Fernández, A., & Coromina, Ò. (2018) · Convergence The International Journal of Research into New Media Technologies · doi:10.1177/1354856517736982 · Cited by 280 · see citing works
Right-Wing YouTube: A Supply and Demand Perspective
Munger, K., & Phillips, J. (2022) · The International Journal of Press/Politics · doi:10.1177/1940161220964767 · Cited by 265 · see citing works
Measuring Misinformation in Video Search Platforms: An Audit Study on YouTube
Hussein, E., Juneja, P., & Mitra, T. (2020) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3392854 · Cited by 229 · see citing works
Auditing Algorithms: Understanding Algorithmic Systems from the Outside In
Metaxa, D., Park, J., Robertson, R. E., Karahalios, K., Wilson, C., Hancock, J., & Sandvig, C. (2021) · Found. Trends Hum. Comput. Interact. · doi:10.1561/1100000083 · Cited by 218 · see citing works
Self-citationAuditing Partisan Audience Bias within Google Search
Robertson, R. E., Jiang, S., Joseph, K., Friedland, L., Lazer, D., & Wilson, C. (2018) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3274417 · Cited by 209 · see citing works
Problematic Machine Behavior: A Systematic Literature Review of Algorithm Audits
Bandy, J. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3449148 · Cited by 193 · see citing works
Zarouali, B., Boerman, S. C., & de Vreese, C. H. (2021) · Telematics and Informatics · doi:10.1016/j.tele.2021.101607 · Cited by 190 · see citing works
Partisan selective exposure in online news consumption: evidence from the 2016 presidential campaign
Peterson, E., Goel, S., & Iyengar, S. (2019) · Political Science Research and Methods · doi:10.1017/psrm.2019.55 · Cited by 156 · see citing works
Users choose to engage with more partisan news than they are exposed to on Google Search
Robertson, R. E., Green, J., Ruck, D. J., Ognyanova, K., Wilson, C., & Lazer, D. (2023) · Nature · doi:10.1038/s41586-023-06078-5 · Cited by 126 · see citing works
Do social media undermine social cohesion? A critical review
González-Bailón, S., & Lelkes, Y. (2022) · Social Issues and Policy Review · doi:10.1111/sipr.12091 · Cited by 110 · see citing works
Exposure to news grows less fragmented with an increase in mobile access
Yang, T., Majó-Vázquez, S., Nielsen, R. K., & González-Bailón, S. (2020) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2006089117 · Cited by 95 · see citing works
van Setten, M., Scheepers, P. L. H., & Lubbers, M. (2017) · European Societies · doi:10.1080/14616696.2016.1268705 · Cited by 79 · see citing works
High level of correspondence across different news domain quality rating sets
Lin, H., Lasser, J., Lewandowsky, S., Cole, R., Gully, A., Rand, D. G., & Pennycook, G. (2023) · PNAS Nexus · doi:10.1093/pnasnexus/pgad286 · Cited by 75 · see citing works
Algorithmic Amplification of Politics on Twitter
Huszár, F., Ktena, S. I., O’Brien, C., Belli, L., Schlaikjer, A., & Hardt, M. (2022) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2025334119 · Cited by 65 · see citing works
Pathak, R., Spezzano, F., & Pera, M. S. (2023) · ACM Transactions on the Web · doi:10.1145/3616088 · Cited by 64 · see citing works
Haroon, M., Wojcieszak, M., Chhabra, A., Liu, X., Mohapatra, P., & Shafiq, Z. (2023) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2213020120 · Cited by 64 · see citing works
Auditing Algorithmic Bias on Twitter
Bartley, N., Abeliuk, A., Ferrara, E., & Lerman, K. (2021) · 13th ACM Web Science Conference 2021 · doi:10.1145/3447535.3462491 · Cited by 62 · see citing works
Auditing YouTube’s Recommendation Algorithm for Misinformation Filter Bubbles
Srba, I., Moro, R., Tomlein, M., Pecher, B., Simko, J., Stefancova, E., Kompan, M., Hrckova, A., Podrouzek, J., Gavornik, A., et al. (2023) · ACM Transactions on Recommender Systems · doi:10.1145/3568392 · Cited by 61 · see citing works
Reply to MTurk, Prolific or panels? Choosing the right audience for online researchPreprint
Litman, L., Moss, A. J., Rosenzweig, C., & Robinson, J. (2021) · SSRN Electronic Journal · doi:10.2139/ssrn.3775075 · Cited by 58 · see citing works
More Accounts, Fewer Links: How Algorithmic Curation Impacts Media Exposure in Twitter Timelines
Bandy, J., & Diakopoulos, N. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3449152 · Cited by 54 · see citing works
Wagner, M. W. (2023) · Science · doi:10.1126/science.adi2430 · Cited by 51 · see citing works
Sociotechnical Audits: Broadening the Algorithm Auditing Lens to Investigate Targeted Advertising
Lam, M. S., Pandit, A., Kalicki, C. H., Gupta, R., Sahoo, P., & Metaxa, D. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3610209 · Cited by 44 · see citing works
Self-citationCurating Quality? How Twitter’s Timeline Algorithm Treats Different Types of News
Bandy, J., & Diakopoulos, N. A. (2021) · Social Media + Society · doi:10.1177/20563051211041648 · Cited by 41 · see citing works
Metaxa, D., Park, J.-S., Landay, J. A., & Hancock, J. T. (2019) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3359231 · Cited by 39 · see citing works
Self-citationAuditing Entertainment Traps on YouTube: How Do Recommendation Algorithms Pull Users Away from News
Huang, S., & Tian, Y. (2024) · Political Communication · doi:10.1080/10584609.2024.2343769 · Cited by 37 · see citing works
Self-citationProceedings of the 13th ACM Web Science Conference 2021
(2021) · doi:10.1145/3447535 · Cited by 13 · see citing works
Zhou, A. Y., Tian, Y., & González‐Bailón, S. (2023) · New Media & Society · doi:10.1177/14614448231196863 · Cited by 12 · see citing works
Self-citationAnalysing the Effect of Recommendation Algorithms on the Amplification of MisinformationPreprint
Fernández, M., Bellogín, A., & Iván Cantador, A. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2103.14748 · Cited by 9 · see citing works
It's All Relative! A Method to Counter Human Bias in Crowdsourced Stance Detection of News Articles
Haq, E.-U., Lu, Y., & Hui, P. (2022) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3555636 · Cited by 8 · see citing works
Just Another Day on Twitter: A Complete 24 Hours of Twitter DataPreprint
Pfeffer, J., Matter, D., Jaidka, K., Varol, O., Mashhadi, A., Lasser, J., Assenmacher, D., Wu, S., Yang, D., Brantner, C., et al. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2301.11429 · Cited by 7 · see citing works
Flagged first, then by citations
Research works printed but not linked: 6
- [7][7] Axel Bruns. 2019. Are filter bubbles real? John Wiley & Sons. -
- [8][8] Michael X. Delli Carpini and Scott Keeter. 1996. What Americans Know about Politics and why it Matters . Yale University Press. Google-Books-ID: b2ZUquOLrzoC. -
- [29][29] Anja Lambrecht and Catherine Tucker. 2019. Algorithmic bias? An empirical study of apparent gender-based discrimination in the display of STEM career ads. Management science 65, 7 (2019), 2966-2981. -
- [31][31] Mark Ledwich and Anna Zaitsev. 2019. Algorithmic extremism: Examining YouTube's rabbit hole of radicalization. arXiv preprint arXiv:1912.11211 (2019). - · arXiv
- [48][48] Manoel Horta Ribeiro, Raphael Ottoni, Robert West, Virgílio A. F. Almeida, and Wagner Meira. 2021. Auditing Radicalization Pathways on YouTube. arXiv:1908.08313 [cs] (Oct. 2021). http://arxiv.org/abs/1908.08313 arXiv: 1908.08313. - · arXiv
- [53][53] Christian Sandvig, Kevin Hamilton, Karrie Karahalios, and Cedric Langbort. 2014. Auditing algorithms: Research methods for detecting discrimination on internet platforms. Data and discrimination: converting critical concerns into productive inquiry 22, 2014 (2014), 4349-4357. -
Cited by
Sorted by citations
Ye, J., Luceri, L., & Ferrara, E. (2025) · Proceedings of the 2025 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3715275.3732159 · Cited by 16 · see citing works
“Future research could address these concerns by 1) systematically comparing in-network and out-of-network exposure biases and 2) conducting user studies to investigate how algorithmically curated timelines influence political attitudes over time (see “sociotechnical audit” [29]).”
InfluentialZhou, A. Y., Metaxa, D., Kim, Y. M., & Jaidka, K. (2024) · Journal of Advertising · doi:10.1080/00913367.2024.2403613 · Cited by 7 · see citing works
Self-citationUrman, A., Makhortykh, M., & Hannák, A. (2025) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3715275.3732026 · Cited by 6 · see citing works
Yu, X., Barnehl, H.-M., & Wojcieszak, M. (2025) · Information Communication & Society · doi:10.1080/1369118x.2025.2565333 · Cited by 4 · see citing works
O’Reilly, D. P., Babata, K., & Roland, D. (2025) · Pediatric Research · doi:10.1038/s41390-025-04233-4
Harliantara, H., & Öztürk, B. (2026) · Frontiers in Communication · doi:10.3389/fcomm.2026.1803100
Riazi, B., de Carvalho, A. F. P., & Culén, A. L. (2026) · Computer Supported Cooperative Work (CSCW) · doi:10.1007/s10606-026-09541-9
Do Language Models Pass the Bechdel Test? Auditing Gender Biases in LLM-Generated Screenplays
Govindu, M. N., Wang, S., Friedler, S. A., & Metaxa, D. (2026) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3805689.3812208
Self-citationFeeling Informed and Feeling Heard: Recommender Systems Usage and Political Efficacy
Wan, J., Araujo, T., Helberger, N., & de Vreese, C. H. (2026) · Journal of Broadcasting & Electronic Media · doi:10.1080/08838151.2026.2705972
Suk, J., Zhong, W., Zhang, Y., Pruden, M., Yang, Y., Mao, C., & DiFranzo, D. J. (2026) · Journal of Information Technology & Politics · doi:10.1080/19331681.2026.2731442
Algorithmic systems, human agency and the future of platform research
Hosseinmardi, H., Dutta, U., Rothschild, D. M., & Watts, D. J. (2026) · Nature Computational Science · doi:10.1038/s43588-026-01038-1
- Authors
- Wang, S., Huang, S., Zhou, A., & Metaxa, D.
- Affiliations
- University of Pennsylvania (US)
- University of Minnesota (US)
- Venue
- Proceedings of the ACM on Human-Computer Interaction
- Published
- 2024-11-07
- Volume
- 8
- Issue
- CSCW2
- Pages
- 1-25
- ISSN
- 2573-0142 (ISSN-L)
- OpenAlex
- W4404172978
- Cited by
- 11
- References
- 42
- Citations by year
- 2026: 6 · 2025: 4 · 2024: 1
- FWCI
- 7.92
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Proceedings of the ACM on Human-Computer Interaction (publishedVersion, cc-by-nc-sa)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- University of Pennsylvania
- Topics
- Social Sciences › Sociology and Political Science › Misinformation and Its Impacts
- Social Sciences › Communication › Social Media and Politics
- Social Sciences › Safety Research › Ethics and Social Impacts of AI
- Keywords
- Twitter, news curation, filter bubbles, echo chambers, ideological segregation, polarization, news quality, news reliability, user perception, civic information
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
