CoChat
Sign Up
Cited by 65Misinformation and Its ImpactsRecommender Systems and TechniquesSpam and Phishing Detection

An Audit of Misinformation Filter Bubbles on YouTube: Bubble Bursting and Recent Behavior Changes

Open access
Tomlein, Matus, Pecher, Branislav, Simko, Jakub, Srba, Ivan, Moro, Robert, Stefancova, Elena, Kompan, Michal, Hrckova, Andrea, Podrouzek, Juraj, Bielikova, Maria·2021·Fifteenth ACM Conference on Recommender Systems
DOI 10.1145/3460231.3474241
Open this paper in CoChat — free accountOpen PDF

Abstract

The negative effects of misinformation filter bubbles in adaptive systems have been known to researchers for some time. Several studies investigated, most prominently on YouTube, how fast a user can get into a misinformation filter bubble simply by selecting “wrong choices” from the items offered. Yet, no studies so far have investigated what it takes to “burst the bubble”, i.e., revert the bubble enclosure. We present a study in which pre-programmed agents (acting as YouTube users) delve into misinformation filter bubbles by watching misinformation promoting content (for various topics). Then, by watching misinformation debunking content, the agents try to burst the bubbles and reach more balanced recommendation mixes. We recorded the search results and recommendations, which the agents encountered, and analyzed them for the presence of misinformation. Our key finding is that bursting of a filter bubble is possible, albeit it manifests differently from topic to topic. Moreover, we observe that filter bubbles do not truly appear in some situations. We also draw a direct comparison with a previous study. Sadly, we did not find much improvements in misinformation occurrences, despite recent pledges by YouTube.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1145/3460231.3474241

Cites

5 of 5 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Most research fails to adequately address dynamic shifts in algorithmic filtering.

    “Although a few studies have conducted continuous audits to examine how algorithms evolve (e.g., [147,166]), most research fails to address these dynamic shifts adequately.”

    From A systematic review of echo chamber research: comparative analysis of conceptualizations, operationalizations, and varying outcomes (2025) · cites this paper as [166]

    The Related Work section (p. 4) says prior YouTube misinformation-filter-bubble studies had not investigated how YouTube’s adaptive systems develop and improve. The Study Design section (p. 5) compares a March 2021 audit with a mid-2019 study, addressing change over time. This supports the point for that research area, but not the broader assertion that most research on algorithmic filtering inadequately addresses dynamic shifts. (LLM, conf=0.82)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 26

  1. 10.1016/j.ijmedinf.2020.104175 · DOI
  2. Eytan Bakshy , Solomon Messing , and Lada  A. Adamic . 2015. Exposure to ideologically diverse news and opinion on Facebook. Science 348, 6239 ( 2015 ), 1130–1132. Eytan Bakshy, Solomon Messing, and Lada A. Adamic. 2015. Exposure to ideologically diverse news and opinion on Facebook. Science 348, 6239 (2015), 1130–1132.
  3. 10.1016/j.chb.2016.08.016 · DOI
  4. 10.1145/2959100.2959190 · DOI · Paper page
  5. Leon Festinger . 1957. A theory of cognitive dissonance. Vol. 2 . Stanford university press . Leon Festinger. 1957. A theory of cognitive dissonance. Vol. 2. Stanford university press.
  6. 10.1145/2488388.2488435 · DOI · Paper page
  7. 10.1145/3392854 · DOI
  8. 10.1145/3411764.3445250 · DOI
  9. Huyen Le , Andrew High , Raven Maragh , Timothy Havens , Brian Ekdale , and Zubair Shafiq . 2019 . Measuring political personalization of Google news search . In Proc. of the World Wide Web Conference (WWW ’19) . 2957–2963. https://doi.org/10.1145/3308558.3312504 Huyen Le, Andrew High, Raven Maragh, Timothy Havens, Brian Ekdale, and Zubair Shafiq. 2019. Measuring political personalization of Google news search. In Proc. of the World Wide Web Conference (WWW ’19). 2957–2963. https://doi.org/10.1145/3308558.3312504 · DOI
  10. 10.1561/100.00016037 · DOI
  11. 10.1145/3359231 · DOI
  12. Diana  C. Mutz and Lori Young . 2011 . Communication and public opinion: Plus ça change?Public opinion quarterly 75, 5 (2011), 1018–1044. Diana C. Mutz and Lori Young. 2011. Communication and public opinion: Plus ça change?Public opinion quarterly 75, 5 (2011), 1018–1044.
  13. Kostantinos Papadamou Savvas Zannettou Jeremy Blackburn Emiliano De Cristofaro Gianluca Stringhini and Michael Sirivianos. 2020. ”It is just a flu”: Assessing the Effect of Watch History on YouTube’s Pseudoscientific Video Recommendations. arxiv:2010.11638 [cs.CY] Kostantinos Papadamou Savvas Zannettou Jeremy Blackburn Emiliano De Cristofaro Gianluca Stringhini and Michael Sirivianos. 2020. ”It is just a flu”: Assessing the Effect of Watch History on YouTube’s Pseudoscientific Video Recommendations. arxiv:2010.11638 [cs.CY] · arXiv
  14. Eli Pariser . 2011. The filter bubble: What the Internet is hiding from you . Penguin UK. Eli Pariser. 2011. The filter bubble: What the Internet is hiding from you. Penguin UK.
  15. Branislav Pecher , Ivan Srba , Robert Moro , Matus Tomlein , and Maria Bielikova . 2021 . FireAnt: Claim-Based Medical Misinformation Detection and Monitoring . In Proc. of the Joint European Conference on Machine Learning and Knowledge Discovery in Databases (ECML PKDD ’20) . 555–559. https://doi.org/10.1007/978-3-030-67670-4_38 Branislav Pecher, Ivan Srba, Robert Moro, Matus Tomlein, and Maria Bielikova. 2021. FireAnt: Claim-Based Medical Misinformation Detection and Monitoring. In Proc. of the Joint European Conference on Machine Learning and Knowledge Discovery in Databases (ECML PKDD ’20). 555–559. https://doi.org/10.1007/978-3-030-67670-4_38 · DOI
  16. Ronald  E. Robertson , David Lazer , and Christo Wilson . 2018 . Auditing the personalization and composition of politically-related search engine results pages . Proc. of the World Wide Web Conference (WWW ’18) , 955–965. https://doi.org/10.1145/3178876.3186143 Ronald E. Robertson, David Lazer, and Christo Wilson. 2018. Auditing the personalization and composition of politically-related search engine results pages. Proc. of the World Wide Web Conference (WWW ’18), 955–965. https://doi.org/10.1145/3178876.3186143 · DOI
  17. Márcio Silva , Lucas Santos de Oliveira , Athanasios Andreou , Pedro Olmo Vaz de Melo , Oana Goga , and Fabricio Benevenuto . 2020 . Facebook Ads Monitor: An Independent Auditing System for Political Ads on Facebook . In Proc. of The Web Conference (WWW ’20) . ACM, New York, NY, USA, 224–234. https://doi.org/10.1145/3366423.3380109 Márcio Silva, Lucas Santos de Oliveira, Athanasios Andreou, Pedro Olmo Vaz de Melo, Oana Goga, and Fabricio Benevenuto. 2020. Facebook Ads Monitor: An Independent Auditing System for Political Ads on Facebook. In Proc. of The Web Conference (WWW ’20). ACM, New York, NY, USA, 224–234. https://doi.org/10.1145/3366423.3380109 · DOI
  18. Jakub Simko Patrik Racsko Matus Tomlein Martina Hanakova Robert Moro and Maria Bielikova. 2021. A study of fake news reading and annotating in social media context. New Review of Hypermedia and Multimedia(2021) 1–31. Jakub Simko Patrik Racsko Matus Tomlein Martina Hanakova Robert Moro and Maria Bielikova. 2021. A study of fake news reading and annotating in social media context. New Review of Hypermedia and Multimedia(2021) 1–31. · DOI
  19. Jakub Simko , Matus Tomlein , Branislav Pecher , Robert Moro , Ivan Srba , Elena Stefancova , Andrea Hrckova , Michal Kompan , Juraj Podrouzek , and Maria Bielikova . 2021. Towards Continuous Automatic Audits of Social Media Adaptive Behavior and Its Role in Misinformation Spreading . In Adjunct Proc. of the 29th ACM Conference on User Modeling, Adaptation and Personalization (UMAP ’21) . ACM , New York, NY, USA , 411–414. https://doi.org/10.1145/3450614.3463353 Jakub Simko, Matus Tomlein, Branislav Pecher, Robert Moro, Ivan Srba, Elena Stefancova, Andrea Hrckova, Michal Kompan, Juraj Podrouzek, and Maria Bielikova. 2021. Towards Continuous Automatic Audits of Social Media Adaptive Behavior and Its Role in Misinformation Spreading. In Adjunct Proc. of the 29th ACM Conference on User Modeling, Adaptation and Personalization (UMAP ’21). ACM, New York, NY, USA, 411–414. https://doi.org/10.1145/3450614.3463353 · DOI
  20. 10.1145/3386392.3399566 · DOI
  21. Ivan Srba , Robert Moro , Daniela Chuda , Maria Bielikova , Jakub Simko , Jakub Sevcech , Daniela Chuda , Pavol Navrat , and Maria Bielikova . 2019 . Monant: Universal and Extensible Platform for Monitoring, Detection and Mitigation of Antisocial Behavior . In Proc. of Workshop on Reducing Online Misinformation Exposure (ROME ’19) . 1–7. Ivan Srba, Robert Moro, Daniela Chuda, Maria Bielikova, Jakub Simko, Jakub Sevcech, Daniela Chuda, Pavol Navrat, and Maria Bielikova. 2019. Monant: Universal and Extensible Platform for Monitoring, Detection and Mitigation of Antisocial Behavior. In Proc. of Workshop on Reducing Online Misinformation Exposure (ROME ’19). 1–7.
  22. 10.2139/ssrn.199668 · DOI
  23. Siva Vaidhyanathan . 2018. Antisocial media: How Facebook disconnects us and undermines democracy . Oxford University Press . Siva Vaidhyanathan. 2018. Antisocial media: How Facebook disconnects us and undermines democracy. Oxford University Press.
  24. 10.1145/3298689.3346997 · DOI
  25. 10.1145/3395046 · DOI · Paper page
  26. Shoshana Zuboff . 2019. The Age of Surveillance Capitalism: The Fight for a Human Future at the New Frontier of Power . Profile Books . Shoshana Zuboff. 2019. The Age of Surveillance Capitalism: The Fight for a Human Future at the New Frontier of Power. Profile Books.

Cited by

Sorted by citations

Authors
Tomlein, M., Pecher, B., Simko, J., Srba, I., Moro, R., Stefancova, E., Kompan, M., Hrckova, A., Podrouzek, J., & Bielikova, M.
Affiliations
  • Kempelen Institute of Intelligent Technologies, Slovakia
  • Kempelen Institute of Intelligent Technologies (SK)
  • Kempelen institute of intelligent technologies, Slovakia
ORCID
Venue
Fifteenth ACM Conference on Recommender Systems
Publisher
ACM
Published
2021-09-13
Pages
1-11
DOI
10.1145/3460231.3474241
OpenAlex
W3200592006
Cited by
65
References
5
Citations by year
2026: 13 · 2025: 8 · 2024: 16 · 2023: 18 · 2022: 10
FWCI
28.27
Type
conference paper
Language
en
Open access
Licence
www.acm.org/publications/policies/copyright_policy#Background (vor)
Locations
  • ACM Conference on Recommender Systems (RecSys) (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Topics
  • Social Sciences › Sociology and Political Science › Misinformation and Its Impacts
  • Physical Sciences › Computer Science › Information Systems › Recommender Systems and Techniques
  • Physical Sciences › Computer Science › Information Systems › Spam and Phishing Detection
Keywords
YouTube, bubble bursting, misinformation debunking, recommender systems, adaptive agents, user behavior, search results, recommendation diversity

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review