Auditing the Partisanship of Google Search Snippets
Open accessThe DOI matches its Crossref record; 6 of the 6 references it cites were found in a registry or an index.
Abstract
The text snippets presented in web search results provide users with a slice of page content that they can quickly scan to help inform their click decisions. However, little is known about how these snippets are generated or how they relate to a user's search query. Motivated by the growing body of evidence suggesting that search engine rankings can influence undecided voters, we conducted an algorithm audit of the political partisanship of Google Search snippets relative to the webpages they are extracted from. To accomplish this, we constructed lexicon of partisan cues to measure partisanship and construct a set of left- and right-leaning search queries. Then, we collected a large dataset of Search Engine Results Pages (SERPs) by running our partisan queries and their autocomplete suggestions on Google Search. After using our lexicon to score the machine-coded partisanship of snippets and webpages, we found that Google Search's snippets generally amplify partisanship, and that this effect is robust across different types of webpages, query topics, and partisan (left- and right-leaning) queries.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Exposure to ideologically diverse news and opinion on Facebook
Bakshy, E., Messing, S., & Adamic, L. A. (2015) · Science · doi:10.1126/science.aaa1160 · Cited by 3,245
Reuters Institute Digital News Report 2015
Newman, N., Levy, D. A. L., & Nielsen, R. K. (2015) · SSRN Electronic Journal · doi:10.2139/ssrn.2619576 · Cited by 1,890
Extracting Policy Positions from Political Texts Using Words as Data
LAVER, M., BENOIT, K., & GARRY, J. (2003) · American Political Science Review · doi:10.1017/s0003055403000698 · Cited by 1,377 · cited 1×
“Using the lexicons, the machine-coded partisanship of a document can be quantified by looking at the number and strength of polarized terms in the document [42, 44].”
BackgroundWhat are you looking for?: an eye-tracking study of information usage in web search
Cutrell, E., & Guan, Z. (2007) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1240624.1240690 · Cited by 460 · cited 2×
“There are studies on whether the length of snippets impact user experience and the informative-ness of search results [13, 38], as well as how snippet length should be customized for mobile web search [37].”
BackgroundFair and Balanced? Quantifying Media Bias through Crowdsourced Content Analysis
Budak, C., Goel, S., & Rao, J. M. (2016) · Public Opinion Quarterly · doi:10.1093/poq/nfw007 · Cited by 336 · cited 2×
“First, the Γ scores that we calculate for individual webpages are not strongly correlated with the overall partisanship of the associated websites, as measured using existing scales like AllSides and others [1, 2, 6, 57].”
BackgroundAuditing Partisan Audience Bias within Google Search
Robertson, R. E., Jiang, S., Joseph, K., Friedland, L., Lazer, D., & Wilson, C. (2018) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3274417 · Cited by 209
Self-citation
Cited by
Problematic Machine Behavior: A Systematic Literature Review of Algorithm Audits
Bandy, J. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3449148 · Cited by 193
Building and Auditing Fair Algorithms: A Case Study in Candidate Screening
Wilson, C., Ghosh, A., Jiang, S., Mislove, A., Baker, L., Szary, J., Trindel, K., & Polli, F. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445928 · Cited by 133
Self-citationMore Accounts, Fewer Links: How Algorithmic Curation Impacts Media Exposure in Twitter Timelines
Bandy, J., & Diakopoulos, N. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3449152 · Cited by 54
“We used a scale of [-1,1], similar to ideological scales in related work [8, 45, 96], with negative values representing left-leaning ideology and positive values representing right-leaning ideology: # 𝐿𝑒𝑔𝑖𝑠𝑙𝑎𝑡𝑜𝑟𝑠 𝐹𝑜𝑙𝑙𝑜𝑤𝑒𝑑 Then, we created two networks and assigned users to them…”
Metadata
- Authors
- Hu, D., Jiang, S., E. Robertson, R., & Wilson, C.
- Venue
- The World Wide Web Conference
- Publisher
- ACM
- Published
- 2019-05-13
- Pages
- 693-704
- OpenAlex
- W2912845403
- Cited by
- 51
- References
- 6
- FWCI
- 5.59
- Type
- conference paper
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Auditing the Partisanship of Google Search Snippets. https://app.cochat.ai/papers/10.1145/3308558.3313654
- The paper
- Hu, D., Jiang, S., E. Robertson, R., & Wilson, C. (2019). Auditing the Partisanship of Google Search Snippets. The World Wide Web Conference. https://doi.org/10.1145/3308558.3313654
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.