The DOI matches its Crossref record; 11 of the 11 references it cites were found in a registry or an index.
Beliefs and biases in web search
Abstract
People's beliefs, and unconscious biases that arise from those beliefs, influence their judgment, decision making, and actions, as is commonly accepted among psychologists. Biases can be observed in information retrieval in situations where searchers seek or are presented with information that significantly deviates from the truth. There is little understanding of the impact of such biases in search. In this paper we study search-related biases via multiple probes: an exploratory retrospective survey, human labeling of the captions and results returned by a Web search engine, and a large-scale log analysis of search behavior on that engine. Targeting yes-no questions in the critical domain of health search, we show that Web searchers exhibit their own biases and are also subject to bias from the search engine. We clearly observe searchers favoring positive information over negative and more than expected given base rates based on consensus answers from physicians. We also show that search engines strongly favor a particular, usually positive, perspective, irrespective of the truth. Importantly, we show that these biases can be counterproductive and affect search outcomes; in our study, around half of the answers that searchers settled on were actually incorrect. Our findings have implications for search engine design, including the development of ranking algorithms that con-sider the desire to satisfy searchers (by validating their beliefs) and providing accurate answers and properly considering base rates. Incorporating likelihood information into search is particularly important for consequential tasks, such as those with a medical focus.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Tversky, A., & Kahneman, D. (1974) · Science · doi:10.1126/science.185.4157.1124 · Cited by 28,810
Availability: A heuristic for judging frequency and probability
Tversky, A., & Kahneman, D. (1973) · Cognitive Psychology · doi:10.1016/0010-0285(73)90033-9 · Cited by 9,984 · cited 2×
“…searchers susceptible to the effects of information availability [33][39], where people’s ability to make rational decisions is limited by the information that they have access to [33] and the ease with which information is recalled (via the engine in this case) maps to likelihood estimates [39].”
BackgroundMeasuring nominal scale agreement among many raters.
Fleiss, J. L. (1971) · Psychological Bulletin · doi:10.1037/h0031619 · Cited by 8,733 · cited 1×
“We found a high level of agreement between the physicians (both κ ≥ 0.886), as well as between the two physicians and the consensus labels provided by the crowd-sourced judges (both Fleiss’ multi-rater κ ≥ 0.853 [13]).”
ResultInfluentialA vector space model for automatic indexing
Salton, G., Wong, A., & Yang, C. S. (1975) · Communications of the ACM · doi:10.1145/361219.361220 · Cited by 7,521 · cited 3×
“Numerous models of the search process have been developed over the past few decades [4][17][23][24][30].”
BackgroundConfirmation, disconfirmation, and information in hypothesis testing
Klayman, J., & Ha, Y.-W. (1987) · Psychological Review · doi:10.1037/0033-295x.94.2.211 · Cited by 2,094 · cited 1×
“This can make searchers more likely to employ positive test strategies, where people seek evidence that supports their hypothesis and disregard evidence that refutes it [22].”
BackgroundFILTER BUBBLE: Wie wir im Internet entmündigt werden
Pariser, E. (2012) · doi:10.3139/9783446431164 · Cited by 1,849
Information Seeking in Electronic Environments
Marchionini, G. (1995) · doi:10.1017/cbo9780511626388 · Cited by 833
Cyberchondria: studies of the escalation of medical concerns in web search
White, R. W., & Horvitz, E. (2009) · ACM Transactions on Information Systems · doi:10.1145/1629096.1629101 · Cited by 546
Self-citationSearch engine coverage bias: evidence and possible causes
Vaughan, L., & Thelwall, M. (2004) · Information Processing & Management · doi:10.1016/s0306-4573(03)00063-3 · Cited by 259
Augmenting web pages and search results to support credibility assessment
Schwarz, J., & Morris, M. (2011) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1978942.1979127 · Cited by 181
The influence of caption features on clickthrough patterns in web search
Clarke, C. L. A., Agichtein, E., Dumais, S., & White, R. W. (2007) · Proceedings of the 30th annual international ACM SIGIR conference on Research and development in information retrieval · doi:10.1145/1277741.1277767 · Cited by 119 · cited 2×
“Studying the role of captions is particularly important as they play a critical role in searchers’ decision making [9][47].”
BackgroundSelf-citation
Cited by
Social Data: Biases, Methodological Pitfalls, and Ethical Boundaries
Olteanu, A., Castillo, C., Diaz, F., & Kiciman, E. (2016) · doi:10.2139/ssrn.2886526 · Cited by 763
Social Data: Biases, Methodological Pitfalls, and Ethical Boundaries
Olteanu, A., Castillo, C., Diaz, F., & Kıcıman, E. (2019) · Frontiers in Big Data · doi:10.3389/fdata.2019.00013 · Cited by 763
Microsoft Academic Graph: When experts are not enough
Wang, K., Shen, Z., Huang, C., Wu, C.-H., Dong, Y., & Kanakia, A. (2020) · Quantitative Science Studies · doi:10.1162/qss_a_00021 · Cited by 509
Baeza-Yates, R. (2018) · Communications of the ACM · doi:10.1145/3209581 · Cited by 432
Cognitive Biases in Crowdsourcing
Eickhoff, C. (2018) · Proceedings of the Eleventh ACM International Conference on Web Search and Data Mining · doi:10.1145/3159652.3159654 · Cited by 144
Metadata
- Authors
- White, R.
- Venue
- Proceedings of the 36th international ACM SIGIR conference on Research and development in information retrieval
- Publisher
- ACM
- Published
- 2013-07-28
- Pages
- 3-12
- OpenAlex
- W2159205954
- Cited by
- 183
- References
- 11
- FWCI
- 8.34
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.