Generative Echo Chamber? Effect of LLM-Powered Search Systems on Diverse Information Seeking
Open accessThe DOI matches its Crossref record; 67 of the 67 references it cites were found in a registry or an index.
Abstract
Large language models (LLMs) powered conversational search systems have already been used by hundreds of millions of people, and are believed to bring many benefits over conventional search. However, while decades of research and public discourse interrogated the risk of search systems in increasing selective exposure and creating echo chambers—limiting exposure to diverse opinions and leading to opinion polarization, little is known about such a risk of LLM-powered conversational search. We conduct two experiments to investigate: 1) whether and how LLM-powered conversational search increases selective exposure compared to conventional search; 2) whether and how LLMs with opinion biases that either reinforce or challenge the user’s view change the effect. Overall, we found that participants engaged in more biased information querying with LLM-powered conversational search, and an opinionated LLM reinforcing their views exacerbated this bias. These results present critical implications for the development of LLMs and conversational search systems, and the policy governing these technologies.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“This impacts the interpretation of our findings on trust, ethics and ownership (Section 8), as the discrepancy between how AI functions (Section 6) and how users perceive its intentionality [17, 145] influences responsibility alignment and bias perception [148, 176] in ways our current framework only partially addresses.”
From Exploring Collaboration Patterns and Strategies in Human-AI Co-creation through the Lens of Agency: A Scoping Review of the Top-tier HCI Literature (2025) · cites this paper as [148]
In the citing paper
“AI systems pose ethical issues including generating inaccuracies [165], violating academic integrity [107], amplifying dominant views [148], and fostering human biases like confirmation bias and over-reliance [107].”
From Exploring Collaboration Patterns and Strategies in Human-AI Co-creation through the Lens of Agency: A Scoping Review of the Top-tier HCI Literature (2025) · cites this paper as [148]
Citations
Cites
Confirmation Bias: A Ubiquitous Phenomenon in Many Guises
Nickerson, R. S. (1998) · Review of General Psychology · doi:10.1037/1089-2680.2.2.175 · Cited by 7,267
On the Opportunities and Risks of Foundation Models
Bommasani, R., Hudson, D. A., Adeli, E., Altman, R., Arora, S., von Arx, S., Bernstein, M. S., Bohg, J., Bosselut, A., Brunskill, E., et al. (2021) · arXiv · doi:10.48550/arxiv.2108.07258 · Cited by 2,676 · cited 1×
“Last but not least, LLMs are known to encode biases from the training data [1, 52], and can be easily steered to exhibit certain opinion biases through model adaptation techniques such as fne-tuning or prompting [6].”
The political blogosphere and the 2004 U.S. election: divided they blog
Adamic, L. A., & Glance, N. (2005) · Proceedings of the 3rd international workshop on Link discovery · doi:10.1145/1134271.1134277 · Cited by 2,619
Generative Agents: Interactive Simulacra of Human Behavior
Park, J. S., O'Brien, J., Cai, C. J., Morris, M. R., Liang, P., & Bernstein, M. S. (2023) · Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology · doi:10.1145/3586183.3606763 · Cited by 2,187
Generative Agents: Interactive Simulacra of Human Behavior
Park, J. S., O'Brien, J. C., Cai, C. J., Morris, M. R., Liang, P., & Bernstein, M. S. (2023) · arXiv · doi:10.48550/arxiv.2304.03442 · Cited by 2,173
Nass, C., Steuer, J., & Tauber, E. R. (1994) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/191666.191703 · Cited by 2,148 · cited 1×
“Prior experience with Conversational AI: People’s prior experience with and attitude towards technology may afect their interaction experience [42].”
The echo chamber effect on social media
Cinelli, M., De Francisci Morales, G., Galeazzi, A., Quattrociocchi, W., & Starnini, M. (2021) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2023301118 · Cited by 2,092
Accommodation theory: Communication, context, and consequence
Giles, H., Coupland, N., & Coupland, J. (1991) · Contexts of Accommodation · doi:10.1017/cbo9780511663673.001 · Cited by 1,442
Festinger, L. (1962) · Scientific American · doi:10.1038/scientificamerican1062-93 · Cited by 1,421 · cited 1×
“Both biases can be attributed to a fundamental desire to avoid or reduce cognitive dissonance [15].”
BackgroundThe law of group polarization.
Sunstein, C. R. (2002) · Journal of Political Philosophy · doi:10.1111/1467-9760.00148 · Cited by 1,371
The filter bubble: how the new personalized web is changing what we read and how we think
(2012) · Choice Reviews Online · doi:10.5860/choice.50-0926 · Cited by 1,319 · cited 2×
“For example, by personalization and supplying only information people want to see, search engines and recommender systems such as news feeds may produce “flter bubbles” [46] of ideological and cultural isolation.”
Proceedings of the 18th ACM Conference on Computer Supported Cooperative Work & Social Computing
Cosley, D., Forte, A., Ciolfi, L., & McDonald, D. W. (2015) · Cited by 1,245
The echo chamber is overstated: the moderating effect of political interest and diverse media
Dubois, E., & Blank, G. (2018) · Information, Communication & Society · doi:10.1080/1369118x.2018.1428656 · Cited by 1,055
Echo chambers online?: Politically motivated selective exposure among Internet news users
Garrett, R. K. (2009) · Journal of Computer-Mediated Communication · doi:10.1111/j.1083-6101.2009.01440.x · Cited by 1,040 · cited 1×
“Other researchers were concerned about the afordances of technologies for people to selectively curate their own information environment, such as by following only or mostly like-minded others on social media platforms [2, 8] or limiting the diversity of the sources for one’s news feed [18].”
BackgroundPerformance Consequences of Automation-Induced 'Complacency'
Parasuraman, R., Molloy, R., & Singh, I. L. (1993) · The International Journal of Aviation Psychology · doi:10.1207/s15327108ijap0301_1 · Cited by 875 · cited 1×
“In the context of co-writing with LMs, common concerns include over-reliance on AI and automation-induced complacency [45] that can lead to not only sub-optimal writing outcomes [3] but also loss of human agency and perceived ownership of the created content [11].”
BackgroundA simple procedure for the assessment of acceptance of advanced transport telematics
Van Der Laan, J. D., Heino, A., & De Waard, D. (1997) · Transportation Research Part C: Emerging Technologies · doi:10.1016/s0968-090x(96)00025-3 · Cited by 873 · cited 1×
“We adapted questions from the Technology Acceptance Model [61] for conversational AI, which measures people’s attitudes towards a system from the perspective of Usefulness (5 items) and Satisfaction (4 items).”
Taxonomy of Risks posed by Language Models
Weidinger, L., Uesato, J., Rauh, M., Griffin, C., Huang, P.-S., Mellor, J., Glaese, A., Cheng, M., Balle, B., Kasirzadeh, A., et al. (2022) · 2022 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3531146.3533088 · Cited by 764 · cited 1×
“These processes can lead to multiple types of harm [56, 64], from discrimination and exclusion, to disinformation and misinformation, as well as creating information hazards such as leaking sensitive information or compromising privacy.”
BackgroundA Survey on Conversational Recommender Systems
Jannach, D., Manzoor, A., Cai, W., & Chen, L. (2022) · ACM Computing Surveys · doi:10.1145/3453154 · Cited by 521 · cited 1×
“While LLM-powered search systems are a recent phenomenon, years of research have pursued the idea of “conversational search”, which allows information retrieval through natural and fexible conversations [26, 37, 51, 67, 70].”
BackgroundPersistent Anti-Muslim Bias in Large Language Models
Abid, A., Farooqi, M., & Zou, J. (2021) · Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3461702.3462624 · Cited by 476 · cited 1×
“Last but not least, LLMs are known to encode biases from the training data [1, 52], and can be easily steered to exhibit certain opinion biases through model adaptation techniques such as fne-tuning or prompting [6].”
BackgroundA Theoretical Framework for Conversational Search
Radlinski, F., & Craswell, N. (2017) · Proceedings of the 2017 Conference on Conference Human Information Interaction and Retrieval · doi:10.1145/3020165.3020183 · Cited by 439 · cited 4×
“Indeed, prior works argued that conversational search and agent systems have the potential advantage of making individuals more receptive toward proactive interactions from the sys-tem [51], and HCI researchers explored leveraging conversational systems such as Alexa to broadcast diverse views [14].”
BackgroundInfluentialSunstein, C. R. (1999) · SSRN Electronic Journal · doi:10.2139/ssrn.199668 · Cited by 384
Wordcraft: Story Writing With Large Language Models
Yuan, A., Coenen, A., Reif, E., & Ippolito, D. (2022) · 27th International Conference on Intelligent User Interfaces · doi:10.1145/3490099.3511105 · Cited by 374
Lee, M., Liang, P., & Yang, Q. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3502030 · Cited by 345
Towards Conversational Search and Recommendation: System Ask, User Respond
Zhang, Y., Chen, X., Ai, Q., Yang, L., & Croft, W. B. (2018) · Proceedings of the 27th ACM International Conference on Information and Knowledge Management · doi:10.1145/3269206.3271776 · Cited by 319
Zhou, J., Zhang, Y., Luo, Q., Parker, A. G., & De Choudhury, M. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581318 · Cited by 288
Draxler, F., Werner, A., Lehmann, F., Hoppe, M., Schmidt, A., Buschek, D., & Welsch, R. (2023) · arXiv · doi:10.48550/arxiv.2303.03283 · Cited by 284
Sociotechnical Harms of Algorithmic Systems: Scoping a Taxonomy for Harm Reduction
Shelby, R., Rismani, S., Henne, K., Moon, A., Rostamzadeh, N., Nicholas, P., Yilla-Akbari, N., Gallegos, J., Smart, A., Garcia, E., et al. (2023) · Proceedings of the 2023 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3600211.3604673 · Cited by 279
Marginally Significant Effects as Evidence for Hypotheses: Changing Attitudes Over Four Decades
Pritschet, L., Powell, D., & Horne, Z. (2016) · Psychological Science · doi:10.1177/0956797616645672 · Cited by 268
Co-Writing with Opinionated Language Models Affects Users’ Views
Jakesch, M., Bhat, A., Buschek, D., Zalmanson, L., & Naaman, M. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581196 · Cited by 265
Ross, S. I., Martinez, F., Houde, S., Muller, M., & Weisz, J. D. (2023) · Proceedings of the 28th International Conference on Intelligent User Interfaces · doi:10.1145/3581641.3584037 · Cited by 258
Scaling Language Models: Methods, Analysis & Insights from Training Gopher
Rae, J. W., Borgeaud, S., Cai, T., Millican, K., Hoffmann, J., Song, F., Aslanides, J., Henderson, S., Ring, R., Young, S., et al. (2021) · arXiv · doi:10.48550/arxiv.2112.11446 · Cited by 235
Presenting diverse political opinions: how and how much
Munson, S. A., & Resnick, P. (2010) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1753326.1753543 · Cited by 233
Supporting reflective public thought with considerit
Kriplean, T., Morgan, J., Freelon, D., Borning, A., & Bennett, L. (2012) · Proceedings of the ACM 2012 conference on Computer Supported Cooperative Work · doi:10.1145/2145204.2145249 · Cited by 224
Investigating Explainability of Generative AI for Code through Scenario-based Design
Sun, J., Liao, Q. V., Muller, M., Agarwal, M., Houde, S., Talamadupula, K., & Weisz, J. D. (2022) · 27th International Conference on Intelligent User Interfaces · doi:10.1145/3490099.3511119 · Cited by 219
Self-citationNewsCube: delivering multiple aspects of news to mitigate media bias
Park, S., Kang, S., Chung, S., & Song, J. (2009) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1518701.1518772 · Cited by 205
Jakesch, M., French, M., Ma, X., Hancock, J. T., & Naaman, M. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300469 · Cited by 190
Draxler, F., Werner, A., Lehmann, F., Hoppe, M., Schmidt, A., Buschek, D., & Welsch, R. (2024) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3637875 · Cited by 186 · cited 1×
“In the context of co-writing with LMs, common concerns include over-reliance on AI and automation-induced complacency [45] that can lead to not only sub-optimal writing outcomes [3] but also loss of human agency and perceived ownership of the created content [11].”
BackgroundKarinshak, E., Liu, S. X., Park, J. S., & Hancock, J. T. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3579592 · Cited by 167
Opinion space: a scalable tool for browsing online comments
Faridani, S., Bitton, E., Ryokai, K., & Goldberg, K. (2010) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1753326.1753502 · Cited by 164
Informing the Design of Spoken Conversational Search: Perspective Paper
Trippas, J. R., Spina, D., Cavedon, L., Joho, H., & Sanderson, M. (2018) · Proceedings of the 2018 Conference on Human Information Interaction&Retrieval - CHIIR '18 · doi:10.1145/3176349.3176387 · Cited by 146
Metaphoria: An Algorithmic Companion for Metaphor Creation
Gero, K. I., & Chilton, L. B. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300526 · Cited by 146
Perfection Not Required? Human-AI Partnerships in Code Translation
Weisz, J. D., Muller, M., Houde, S., Richards, J., Ross, S. I., Martinez, F., Agarwal, M., & Talamadupula, K. (2021) · 26th International Conference on Intelligent User Interfaces · doi:10.1145/3397481.3450656 · Cited by 129 · cited 1×
“…with them before this wave of widely adopted LLMs. Popular applications of LMs include code generation as programming assistance [55, 57, 65], various forms of writing assistance such as next sentence generation [25, 31], summarization of documents [10], rewriting [69], and metaphor…”
BackgroundSerendipity as an emerging design principle of the infosphere: challenges and opportunities
Reviglio, U. (2019) · Ethics and Information Technology · doi:10.1007/s10676-018-9496-y · Cited by 122
Exploring Conversational Search With Humans, Assistants, and Wizards
Vtyurina, A., Savenkov, D., Agichtein, E., & Clarke, C. L. A. (2017) · Proceedings of the 2017 CHI Conference Extended Abstracts on Human Factors in Computing Systems · doi:10.1145/3027063.3053175 · Cited by 117 · cited 1×
“Empirical studies of human interaction with a conversational search system were relatively limited [4, 60, 62], and often relied on wizard-of-oz approaches.”
BackgroundMark my words!: linguistic style accommodation in social media
Danescu-Niculescu-Mizil, C., Gamon, M., & Dumais, S. (2011) · Proceedings of the 20th international conference on World wide web · doi:10.1145/1963405.1963509 · Cited by 113
Liao, Q. V., Davis, M., Geyer, W., Muller, M., & Shami, N. S. (2016) · Proceedings of the 2016 ACM Conference on Designing Interactive Systems · doi:10.1145/2901790.2901842 · Cited by 92
Self-citationLiao, Q. V., & Fu, W.-T. (2013) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/2470654.2481326 · Cited by 90
Self-citationPredictive text encourages predictable writing
Arnold, K. C., Chauncey, K., & Gajos, K. Z. (2020) · Proceedings of the 25th International Conference on Intelligent User Interfaces · doi:10.1145/3377325.3377523 · Cited by 87
Beyond Text Generation: Supporting Writers with Continuous Automatic Text Summaries
Dang, H., Benharrak, K., Lehmann, F., & Buschek, D. (2022) · Proceedings of the 35th Annual ACM Symposium on User Interface Software and Technology · doi:10.1145/3526113.3545672 · Cited by 84 · cited 1×
“…applications of LMs include code generation as programming assistance [55, 57, 65], various forms of writing assistance such as next sentence generation [25, 31], summarization of documents [10], rewriting [69], and metaphor generation [20], as well as chatbots [44, 66] and social agents [47].”
BackgroundInform the Uninformed: Improving Online Informed Consent Reading with an AI-Powered Chatbot
Xiao, Z., Li, T. W., Karahalios, K. G., & Sundaram, H. (2023) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581252 · Cited by 81
Self-citationConversational Information Seeking
Zamani, H., Trippas, J. R., Dalton, J., & Radlinski, F. (2023) · Foundations and Trends® in Information Retrieval · doi:10.1561/1500000081 · Cited by 79
Bias in Search and Recommender Systems
Baeza-Yates, R. (2020) · Fourteenth ACM Conference on Recommender Systems · doi:10.1145/3383313.3418435 · Cited by 76 · cited 1×
“In short, information technologies can have multiplex mechanisms to exacerbate people’s selective exposure bias, from data and algorithmic biases, to biases induced by interaction afordances [5].”
BackgroundResisting Political Fragmentation on the Internet
Garrett, R. K., & Resnick, P. (2011) · Daedalus · doi:10.1162/daed_a_00118 · Cited by 74 · cited 1×
“As Gar-rett and Resnick [19] argued, to increase people’s consumption of attitude-challenging information, the key lies in presenting high-quality challenging items in the right context, and/or reducing people’s cognitive dissonance.”
Characteristics of Harmful Text: Towards Rigorous Benchmarking of Language ModelsPreprint
Rauh, M., Mellor, J. F. J., Uesato, J., Huang, P.-S., Welbl, J., Weidinger, L., Dathathri, S., Glaese, A., Irving, G., Gabriel, I., et al. (2022) · Neural Information Processing Systems · doi:10.48550/arxiv.2206.08325 · Cited by 72
Understanding and controlling the filter bubble through interactive visualization: a user study
Nagulendra, S., & Vassileva, J. (2014) · Proceedings of the 25th ACM conference on Hypertext and social media · doi:10.1145/2631775.2631811 · Cited by 69
Gao, M., Xiao, Z., Karahalios, K., & Fu, W.-T. (2018) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3274324 · Cited by 69
Self-citationKeyvan, K., & Huang, J. X. (2023) · ACM Computing Surveys · doi:10.1145/3534965 · Cited by 69
Escape the bubble: guided exploration of music preferences for serendipity and novelty
Taramigkou, M., Bothos, E., Christidis, K., Apostolou, D., & Mentzas, G. N. (2013) · ACM Conference on Recommender Systems (RecSys) · doi:10.1145/2507157.2507223 · Cited by 68
Promoting Civil Discourse Through Search Engine Diversity
Yom-Tov, E., Dumais, S., & Guo, Q. (2014) · Social Science Computer Review · doi:10.1177/0894439313506838 · Cited by 67 · cited 1×
“Many systems were developed to help people encounter diverse perspectives [14, 68], deliberate on controversial topics [13, 30, 48], be aware of one’s own information bubble and better control fltering mechanisms [17, 27, 41].”
BackgroundPowering an AI Chatbot with Expert Sourcing to Support Credible Health Information Access
Xiao, Z., Liao, Q. V., Zhou, M., Grandison, T., & Li, Y. (2023) · Proceedings of the 28th International Conference on Intelligent User Interfaces · doi:10.1145/3581641.3584031 · Cited by 59
Self-citationJeon, Y., Kim, B., Xiong, A., Lee, D., & Han, K. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3479859 · Cited by 56
Liao, Q. V., & Fu, W.-T. (2014) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/2556288.2557240 · Cited by 47
Self-citationIt Is All About Perspective: An Exploration of Mitigating Selective Exposure with Aspect Indicators
Liao, Q. V., Fu, W., & Mamidi, S. S. (2015) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/2702123.2702570 · Cited by 33
Self-citationUnderstanding User Perception of Automated News Generation System
Oh, C., Choi, J., Lee, S., Park, S., Kim, D., Song, J., Kim, D., Lee, J., & Suh, B. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376811 · Cited by 32
The Effects of System Initiative during Conversational Collaborative Search
Avula, S., Choi, B., & Arguello, J. (2022) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3512913 · Cited by 32
Broadening Exposure to Socio-Political Opinions via a Pushy Smart Home Device
Feltwell, T., Wood, G., Brooker, P., Rowland, S., Baumer, E. P. S., Long, K. S., Vines, J., Barnett, J. C., & Lawson, S. W. (2020) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376774 · Cited by 19 · cited 2×
“Indeed, prior works argued that conversational search and agent systems have the potential advantage of making individuals more receptive toward proactive interactions from the sys-tem [51], and HCI researchers explored leveraging conversational systems such as Alexa to broadcast diverse views [14].”
BackgroundCharacteristics of Harmful Text: Towards Rigorous Benchmarking of Language Models
Rauh, M., Mellor, J., Uesato, J., Huang, P.-S., Welbl, J., Weidinger, L., Dathathri, S., Glaese, A., Irving, G., Gabriel, I., et al. (2022) · neural information processing systems · doi:10.52202/068431-1793 · Cited by 14
Research works printed but not linked: 9
- Michael Conover, Jacob Ratkiewicz, Matthew Francisco, Bruno Gonçalves, Filippo Menczer, and Alessandro Flammini. 2011. Political polarization on twitter. In Proceedings of the international aaai conference on web and social media, Vol. 5. 89–96.
- Dieter Frey. 1986. Recent research on selective exposure to information. Advances in experimental social psychology 19 (1986), 41–80.
- Andrew Guess, Brendan Nyhan, Benjamin Lyons, and Jason Reifler. 2018. Avoiding the echo chamber about echo chambers. Knight Foundation 2, 1 (2018), 1–25.
- William Hart, Dolores Albarracín, Alice H Eagly, Inge Brechan, Matthew J Lindberg, and Lisa Merrill. 2009. Feeling validated versus being correct: a meta-analysis of selective exposure to information.Psychological bulletin 135, 4 (2009), 555.
- Q Vera Liao, Werner Geyer, Michael Muller, and Yasaman Khazaen. 2020. Conversational interfaces for information search. Understanding and Improving Information Search: A Cognitive Approach (2020), 267–287.
- Q Vera Liao and Ziang Xiao. 2023. Rethinking Model Evaluation as Narrowing the Socio-Technical Gap. arXiv preprint arXiv:2306.03100 (2023). · arXiv
- Nelson F. Liu, Tianyi Zhang, and Percy Liang. 2023. Evaluating Verifiability in Generative Search Engines. In The 2023 Conference on Empirical Methods in Natural Language Processing. https://openreview.net/forum?id=ZQV5iRPAua
- Lawrence Philips. 2000. The double metaphone search algorithm. C/C++ users journal 18, 6 (2000), 38–43.
- Peter C Wason. 1960. On the failure to eliminate hypotheses in a conceptual task. Quarterly journal of experimental psychology 12, 3 (1960), 129–140.
Cited by
Bias and Unfairness in Information Retrieval Systems: New Challenges in the LLM Era
Dai, S., Chen, X., Xu, S., Pang, L., Dong, Z., & Xu, J. (2024) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3637528.3671458 · Cited by 331
Accuracy and Political Bias of News Source Credibility Ratings by Large Language Models
Yang, K.-C., & Menczer, F. (2023) · arXiv · doi:10.48550/arxiv.2304.00228 · Cited by 115
Large Language Model‐Based Chatbots in Higher Education
Yigci, D., Eryilmaz, M., Yetisen, A. K., Tasoglu, S., & Ozcan, A. (2025) · Advanced Intelligent Systems · doi:10.1002/aisy.202400429 · Cited by 113
Accuracy and Political Bias of News Source Credibility Ratings by Large Language Models
Yang, K.-C., & Menczer, F. (2025) · Proceedings of the 17th ACM Web Science Conference 2025 · doi:10.1145/3717867.3717903 · Cited by 81
“Furthermore, experiments conducted by Sharma, Liao, and Xiao (2024) demonstrate that users may engage with more biased information when interacting with AI search engines, and LLMs with predefined opinions can exacerbate this bias.”
Kim, S. S. Y., Vaughan, J. W., Liao, Q. V., Lombrozo, T., & Russakovsky, O. (2025) · Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3714020 · Cited by 58
“Already millions of people use LLMs to find answers to their questions about health, science, current events, and other domains, and the use of LLMs is widely predicted to grow [54, 106, 136].”
Self-citation
Metadata
- Authors
- Sharma, N., Liao, Q. V., & Xiao, Z.
- Affiliations
- Johns Hopkins University (US)
- Microsoft (Canada) (CA)
- Venue
- Proceedings of the CHI Conference on Human Factors in Computing Systems
- Publisher
- ACM
- Published
- 2024-05-11
- Pages
- 1-17
- OpenAlex
- W4396832250
- Semantic Scholar
- 0b26abcbb54394c79234eeefe9cb1da5f183d47b
- Cited by
- 132
- References
- 58
- Citations by year
- 2026: 76 · 2025: 50 · 2024: 6
- FWCI
- 50.21
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- CHI Conference on Human Factors in Computing Systems (publishedVersion, cc-by)
- Topics
- Physical Sciences › Computer Science › Information Systems › Information Retrieval and Search Behavior
- Social Sciences › Sociology and Political Science › Misinformation and Its Impacts
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Keywords
- large language models, conversational search, selective exposure, echo chambers, information seeking, opinion polarization, LLM bias, information diversity
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.