CoChat
Sign Up
Cited by 96Misinformation and Its ImpactsInformation Retrieval and Search BehaviorMedia Influence and Politics

Quantifying Biases in Online Information Exposure

Open access
Nikolov, Dimitar, Lalmas, Mounia, Flammini, Alessandro, Menczer, Filippo·2019·Journal of the Association for Information Science and Technology
DOI 10.1002/asi.24121
Open this paper in CoChat — free accountOpen PDF

Abstract

Our consumption of online information is mediated by filtering, ranking, and recommendation algorithms that introduce unintentional biases as they attempt to deliver relevant and engaging content. It has been suggested that our reliance on online technologies such as search engines and social media may limit exposure to diverse points of view and make us vulnerable to manipulation by disinformation. In this article, we mine a massive data set of web traffic to quantify two kinds of bias: (i) homogeneity bias, which is the tendency to consume content from a narrow set of information sources, and (ii) popularity bias, which is the selective exposure to content from top sites. Our analysis reveals different bias levels across several widely used web platforms. Search exposes users to a diverse set of sources, while social media traffic tends to exhibit high popularity and homogeneity bias. When we focus our analysis on traffic to news sites, we find higher levels of popularity bias, with smaller differences across applications. Overall, our results quantify the extent to which our choices of online systems confine us inside “social bubbles.”

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1002/asi.24121

Cites

3 of 3 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Most current digital-media algorithms strongly optimize for engagement.

    “Most current digital-media algorithms strongly optimize for engagement (Narayanan, 2023b; Nikolov et al., 2019).”

    From Social Drivers and Algorithmic Mechanisms on Digital Media. (2024) · cites this paper as Nikolov et al., 2019

    The Introduction (p. 3) says Web-platform algorithms steer attention toward engaging content and that some filtering depends on engagement metrics. It does not establish that most current digital-media algorithms strongly optimize for engagement; it also names relevance, timeliness, reputation, and user preferences as factors. (LLM, conf=0.86)

    Experimental — accuracy not yet measured

  • Users often consume partisan content, reinforcing existing beliefs.

    “Users often consume partisan content, reinforcing existing beliefs [24,54].”

    From A systematic review of echo chamber research: comparative analysis of conceptualizations, operationalizations, and varying outcomes (2025) · cites this paper as [54]

    The Background (p. 5) reports that online users are somewhat more likely to encounter news aligned with their worldviews, supporting the selective-exposure part of the claim. But the Discussion (p. 15) says this study did not analyze content or the political bias of news sources, and it does not establish that consuming such content reinforces users’ beliefs. (LLM, conf=0.86)

    Experimental — accuracy not yet measured

  • Content bias varies by platform, suggesting the impact of platform design on echo chamber formation.

    “Nikolov et al. [54] also observe that content bias varies by platform, suggesting the impact of platform design on echo chamber formation.”

    From A systematic review of echo chamber research: comparative analysis of conceptualizations, operationalizations, and varying outcomes (2025) · cites this paper as [54]

    The Discussion (p. 15) reports differences in exposure bias across platforms and suggests that differences in platform form and function may help explain them. But the authors say they did not examine content or specific platform mechanisms; the Discussion (p. 17) describes echo-chamber formation as a possible consequence of exposure bias, not a demonstrated effect of platform design. (LLM, conf=0.88)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 39

  1. Adamic, L., & Glance, N. (2005). The Political Blogosphere and the 2004 U.S. Election: Divided They Blog. In Proc. 3rd international workshop on link discovery (linkkdd) (pp. 36-43).
  2. An, J., Quercia, D., Cha, M., Gummadi, K., & Crowcroft, J. (2013). Traditional media seen from social media. In Proceedings of the 5th annual acm web science conference (pp. 11-14).
  3. An, J., Quercia, D., & Crowcroft, J. (2013a). Fragmented social media: A look into selective exposure to political news. In Proceedings of the 22nd international conference on world wide web (pp. 51-52). Retrieved from http://dl.acm.org/citation.cfm?id=2487788.2487807
  4. An, J., Quercia, D., & Crowcroft, J. (2013b). Why individuals seek diverse opinions (or why they don't). In Proceedings of the 5th annual acm web science conference (pp. 15-18).
  5. Bakshy, E., Messing, S., & Adamic, L. (2015). Exposure to ideologically diverse news and opinion on facebook. Science , 348 (6239), 1130-1132. doi: 10.1126/science.aaa1160 · DOI · Paper page
  6. Brin, S., & Page, L. (1998, April). The anatomy of a large-scale hypertextual web search engine. Comput. Netw. ISDN Syst. , 30 (1-7), 107-117. Retrieved from http://dx.doi.org/10.1016/S0169-7552(98)00110-X doi: 10.1016/S0169-7552(98)00110-X · DOI · Paper page
  7. Broder, A., Kumar, R., Maghoul, F., Raghavan, P., Rajagopalan, S., Stata, R., . . . Wiener, J. (2000). Graph structure in the web. Computer Networks: The International Journal of Computer and Telecommunications Networking , 33 (1-6), 309-320.
  8. Cho, J., & Roy, S. (2004). Impact of search engines on page popularity. In Proceedings of the 13th international conference on world wide web (pp. 20-29).
  9. Conover, M., Ratkiewicz, J., Francisco, M., Gonçalves, B., Flammini, A., & Menczer, F. (2011). Political Polarization on Twitter. In Proc. 5th international aaai conference on weblogs and social media (icwsm). Retrieved from http://www.aaai.org/ocs/ index.php/ICWSM/ICWSM11/paper/view/2847
  10. Das, A. S., Datar, M., Garg, A., & Rajaram, S. (2007). Google news personalization: Scalable online collaborative filtering. In Proceedings of the 16th international conference on world wide web (pp. 271-280). New York, NY, USA: ACM. Retrieved from http://doi.acm.org/10.1145/1242572.1242610 doi: 10.1145/1242572.1242610 · DOI
  11. Doris-Down, A., Versee, H., & Gilbert, E. (2013). Political blend: An application designed to bring people together based on political differences. In Proceedings of the 6th international conference on communities and technologies (pp. 120-130). ACM.
  12. Fleder, D. M., Hosanagar, K., & Buja, A. (2010). Recommender systems and their effects on consumers: the fragmentation debate. In Proceedings of the 11th acm conference on electronic commerce (ec) (pp. 229-230).
  13. Fortunato, S., Boguna, M., Flammini, A., & Menczer, F. (2007). On local estimations of pagerank: A mean field approach. Internet Mathematics , 4 (2-3), 245-266. Retrieved from http://carl.cs.indiana.edu/fil/Papers/im.pdf doi: 10.1080/15427951.2007.10129294 · DOI
  14. Fortunato, S., Flammini, A., Menczer, F., & Vespignani, A. (2006). Topical interests and the mitigation of search engine bias. Proceedings of the National Academy of Sciences , 103 (34), 12684-12689.
  15. Gilbert, E., Bergstrom, T., & Karahalios, K. (2009). Blogs are echo chambers: Blogs are echo chambers. In Proceedings of 42nd hawaii international conference on system sciences (hicss) (pp. 1-10).
  16. Glenski, M., & Weninger, T. (2017). Rating effects on social news posts and comments. ACM Trans. Intelligent Systems and Tech , 8 (6), 78:1-78:19.
  17. Graells-Garrido, E., Lalmas, M., & Baeza-Yates, R. (2015). Finding intermediary topics between people of opposing views: A case study. In Sigir international workshop on social personalisation and search (sps). Retrieved from http://arxiv.org/abs/1506.00963 http://arxiv.org/abs/1506.00963 · arXiv
  18. Graells-Garrido, E., Lalmas, M., & Quercia, D. (2013). Data portraits: Connecting people of opposing views. (Preprint No. 1311.4658). arXiv. Retrieved from https://arxiv.org/abs/1311.4658 · arXiv
  19. Grevet, C., Terveen, L. G., & Gilbert, E. (2014). Managing political differences in social media. In Proceedings of the 17th acm conference on computer supported cooperative work & social computing (pp. 1400-1408).
  20. Hindman, M., Tsioutsiouliklis, K., & Johnson, J. A. (2003). Googlearchy: How a few heavily-linked sites dominate politics on the web. In Proc. annual meeting of the midwest political science association.
  21. Hosanagar, K., Fleder, D., Lee, D., & Buja, A. (2013). Will the global village fracture into tribes? recommender systems and their effects on consumer fragmentation. Management Science , 60 (4), 805-823.
  22. Introna, L., & Nissenbaum, H. (2000). Defining the web: The politics of search engines. IEEE Computer , 33 (1), 54-62.
  23. Lazer, D. M. J., Baum, M. A., Benkler, Y., Berinsky, A. J., Greenhill, K. M., Menczer, F., . . . Zittrain, J. L. (2018). The science of fake news. Science , 359 (6380), 1094-1096. doi: 10.1126/science.aao2998 · DOI · Paper page
  24. Lehmann, J., Castillo, C., Lalmas, M., & Baeza-Yates, R. A. (2017). Story-focused reading in online news and its potential for user engagement. JASIST , 68 (4), 869-883. Retrieved from https://doi.org/10.1002/asi.23707 doi: 10.1002/asi.23707 · DOI
  25. Linden, G., Smith, B., & York, J. (2003, January). Amazon.com recommendations: Item-to-item collaborative filtering. IEEE Internet Computing , 7 (1), 76-80. Retrieved from http://dx.doi.org/10.1109/MIC.2003.1167344 doi: 10.1109/MIC.2003.1167344 · DOI · Paper page
  26. Lorenz, M. O. (1905). Methods of measuring the concentration of wealth. Publications of the American Statistical Association , 9 (70), 209-219. Retrieved from http://www.jstor.org/stable/2276207
  27. McKenzie, C. R. (2004). Hypothesis testing and evaluation. In D. J. Koehler & N. Harvey (Eds.), Handbook of judgment and decision making (pp. 200-219). Oxford: Blackwell.
  28. Meiss, M., Goncalves, B., Ramasco, J., Flammini, A., & Menczer, F. (2010). Modeling traffic on the web graph. In R. Kumar & D. Sivakumar (Eds.), Proc. 7th workshop on algorithms and models for the web graph (waw) (Vol. 6516, pp. 50-61). Springer.
  29. Mowshowitz, A., & Kawaguchi, A. (2002). Bias on the web. Communications of the ACM , 45 (9), 56-60.
  30. Munson, S. A., & Resnick, P. (2010). Presenting diverse political opinions: How and how much. In Proceedings of the sigchi conference on human factors in computing systems (pp. 1457-1466). ACM.
  31. Nikolov, D., Oliveira, D. F. M., Flammini, A., & Menczer, F. (2015). Measuring online social bubbles. PeerJ Computer Science , 1 (e38). Retrieved from https://dx.doi.org/10.7717/peerj-cs.38 · DOI · Paper page
  32. Pariser, E. (2011). The filter bubble: How the new personalized web is changing what we read and how we think . Penguin.
  33. Ratkiewicz, J., Conover, M., Meiss, M., Gonçalves, B., Flammini, A., & Menczer, F. (2011). Detecting and tracking political abuse in social media. In Proc. 5th international aaai conference on weblogs and social media (icwsm). Retrieved from http:// www.aaai.org/ocs/index.php/ICWSM/ICWSM11/paper/view/2850
  34. Saez-Trumper, D., Castillo, C., & Lalmas, M. (2013). Social media news communities: gatekeeping, coverage, and statement bias. In Proceedings of the 22nd acm international conference on conference on information & knowledge management (cikm) (pp. 1679-1684). doi: 10.1145/2505515.2505623 · DOI
  35. Shao, C., Ciampaglia, G. L., Flammini, A., & Menczer, F. (2016). Hoaxy: A platform for tracking online misinformation. In Proceedings of the 25th international conference companion on world wide web (www) (pp. 745-750). doi: 10.1145/2872518.2890098 · DOI
  36. Stanovich, K. E., West, R. F., & Toplak, M. E. (2013). Myside bias, rational thinking, and intelligence. Current Directions in Psychological Science , 22 (4), 259-264.
  37. Sunstein, C. R. (2002). The law of group polarization. Journal of Political Philosophy , 10 (2), 175-195. Retrieved from http://dx.doi.org/10.1111/1467-9760.00148 doi: 10.1111/1467-9760.00148 · DOI · Paper page
  38. Sunstein, C. R. (2017). # republic.com . Princeton University Press.
  39. West, R., Weber, I., & Castillo, C. (2012). Drawing a data-driven portrait of Wikipedia editors. In Proc. eighth annual international symposium on wikis and open collaboration (wikisym). doi: 10.1145/2462932.2462937 · DOI

Cited by

Sorted by citations

Authors
Nikolov, D., Lalmas, M., Flammini, A., & Menczer, F.
Affiliations
  • Indiana University Bloomington (US)
  • Yahoo Research @ Oath
Corresponding author
Nikolov, Dimitar
ORCID
Venue
Journal of the Association for Information Science and Technology
Publisher
Wiley
Published
2018-11-22
Volume
70
Issue
3
Pages
218-229
ISSN
  • 2330-1635 (ISSN-L)
  • 2330-1643
DOI
10.1002/asi.24121
OpenAlex
W2886256382
Cited by
96
References
3
Citations by year
2026: 6 · 2025: 11 · 2024: 14 · 2023: 17 · 2022: 9 · 2021: 25 · 2020: 6 · 2019: 6 · 2018: 2
FWCI
22.18
Type
journal article
Language
en
Open access
Licence
onlinelibrary.wiley.com/termsAndConditions#vor (vor)
Locations
  • Journal of the Association for Information Science and Technology (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • IUScholarWorks Open (Indiana University) (submittedVersion)
  • arXiv (Cornell University)
Topics
  • Social Sciences › Sociology and Political Science › Misinformation and Its Impacts
  • Physical Sciences › Computer Science › Information Systems › Information Retrieval and Search Behavior
  • Social Sciences › Sociology and Political Science › Media Influence and Politics
Keywords
popularity bias, algorithmic filtering, recommendation algorithms, search engines, social media, selective exposure, echo chambers, social bubbles, disinformation, website traffic, information diversity

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review