The DOI matches its Crossref record; 12 of the 12 references it cites were found in a registry or an index.
Effects of LLM-based Search on Decision Making: Speed, Accuracy, and Overreliance
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Non-conversational LLM-based tools show similar efficiency gains.
“Non-conversational LLM-based tools show similar efficiency gains (Spatharioti et al., 2025).”
From AI Overviews in Academic Search: Evaluating AI-generated Summaries of Search Results in a Domain-specific Search Engine (2026) · cites this paper as Spatharioti et al., 2025
Nonconversational LLM-based tools that accelerate task completion associate with enhanced perceived search experiences.
“Nonconversational LLM-based tools that accelerate task completion associate with enhanced perceived search experiences (Spatharioti et al., 2025).”
From AI Overviews in Academic Search: Evaluating AI-generated Summaries of Search Results in a Domain-specific Search Engine (2026) · cites this paper as Spatharioti et al., 2025
Citations
Cites
Payne, J. W., Bettman, J. R., & Johnson, E. J. (1993) · doi:10.1017/cbo9781139173933 · Cited by 3,487
Experimental evidence on the productivity effects of generative artificial intelligence
Noy, S., & Zhang, W. (2023) · Science · doi:10.1126/science.adh2586 · Cited by 2,150
On Faithfulness and Factuality in Abstractive Summarization
Maynez, J., Narayan, S., Bohnet, B., & McDonald, R. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.173 · Cited by 1,185
Buçinca, Z., Malaya, M. B., & Gajos, K. Z. (2021) · arXiv · doi:10.48550/arxiv.2102.09692 · Cited by 1,124
Dell'Acqua, F., McFowland III, E., Mollick, E. R., Lifshitz-Assaf, H., Kellogg, K., Rajendran, S., Krayer, L., Candelon, F., & Lakhani, K. R. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4573321 · Cited by 794
Understanding the Effect of Accuracy on Trust in Machine Learning Models
Yin, M., Wortman Vaughan, J., & Wallach, H. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300509 · Cited by 527 · cited 1×
“The problem of overreliance on AI has been a concern both before [31, 34, 34, 39, 43, 50] and after the current interest in LLM-generated errors [21, 38, 49].”
The Metacognitive Demands and Opportunities of Generative AI
Tankelevitch, L., Kewenig, V., Simkute, A., Scott, A. E., Sarkar, A., Sellen, A., & Rintel, S. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642902 · Cited by 493 · cited 1×
“Conversely, unreliable output can have metacognitive effects on if and how users choose to use a system or tool [47].”
Productivity Assessment of Neural Code Completion
Ziegler, A., Kalliamvakou, E., Li, X. A., Rice, A., Rifkin, D., Simister, S., Sittampalam, G., & Aftandilian, E. (2022) · Proceedings of the 6th ACM SIGPLAN International Symposium on Machine Programming · doi:10.1145/3520312.3534864 · Cited by 200
Productivity Assessment of Neural Code Completion
Ziegler, A., Kalliamvakou, E., Simister, S., Sittampalam, G., Li, A., Rice, A., Rifkin, D., & Aftandilian, E. (2022) · arXiv · doi:10.48550/arxiv.2205.06537 · Cited by 200
Kim, S. S. Y., Liao, Q. V., Vorvoreanu, M., Ballard, S., & Vaughan, J. W. (2024) · The 2024 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3630106.3658941 · Cited by 147 · cited 2×
“More recent work has explored using natural language to communicate uncertainty (e.g., “I’m not sure, but...”), finding that such approaches reduce overre-liance, although the exact wording impacts effectiveness [21].”
Evaluating Verifiability in Generative Search Engines
Liu, N. F., Zhang, T., & Liang, P. (2023) · arXiv · Cited by 143 · cited 3×
“Despite the promise of LLMs for improving the speed and quality of knowledge work mentioned above, their use is not without risk, particularly when they produce erroneous or fabricated outputs [30, 32].”
Anchoring Bias Affects Mental Model Formation and User Reliance in Explainable AI Systems
Nourani, M., Roy, C., Block, J. E., Honeycutt, D. R., Rahman, T., Ragan, E., & Gogate, V. (2021) · 26th International Conference on Intelligent User Interfaces · doi:10.1145/3397481.3450639 · Cited by 107 · cited 3×
“Participants in both conditions saw this item in the last position (task 5), making it the task where overreliance was more likely to occcur [34].”
Flagged first, then by citations
Research works printed but not linked: 41
- Eugene Agichtein Steve Lawrence and Luis Gravano. 2004. Learning to find answers to questions on the web. ACM Transactions on Internet Technology (TOIT) 4 2 (2004) 129–162. · DOI
- Marcia J Bates. 1989. The design of browsing and berrypicking techniques for the online search interface. Online review 13 5 (1989) 407–424. Publisher: MCB UP Ltd. · DOI
- Michael Bendersky and W. Bruce Croft. 2008. Discovering key concepts in verbose queries(SIGIR ’08). Association for Computing Machinery New York NY USA 491–498. 10.1145/1390334.1390419 · DOI
- 10.1145/2348283.2348312 · DOI
- Sergey Brin. 1998. The PageRank citation ranking: bringing order to the web. Proceedings of ASIS 1998 98 (1998) 161–172.
- Bart J Bronnenberg Jun B Kim and Carl F Mela. 2016. Zooming in on choice: How do consumers search for cameras online? Marketing Science 35 5 (2016) 693–712. · DOI
- 10.3386/w31161 · DOI · Paper page
- 10.1145/3613904.3641904 · DOI
- 10.1145/1963405.1963413 · DOI
- 10.1145/365024.365116 · DOI
- 10.1145/371920.372094 · DOI
- Simon Gächter Esther Kaiser and Manfred Königstein. 2024. Incentive contracts crowd out voluntary cooperation: Evidence from gift-exchange experiments. (2024). · DOI
- Manish Gupta and Michael Bendersky. 2015. Information Retrieval with Verbose Queries(SIGIR ’15). Association for Computing Machinery New York NY USA 1121–1124. · DOI
- 10.1145/2661829.2661912 · DOI
- Bernard J Jansen Amanda Spink and Tefko Saracevic. 2000. Real life real users and real needs: a study and analysis of user queries on the web. Information Processing & Management 36 2 (2000) 207–227. · DOI
- 10.1145/2600428.2609633 · DOI
- Saurav Kadavath Tom Conerly Amanda Askell Tom Henighan Dawn Drain Ethan Perez Nicholas Schiefer Zac Hatfield-Dodds Nova DasSarma Eli Tran-Johnson and others. 2022. Language models (mostly) know what they know. arXiv preprint arXiv:https://arXiv.org/abs/2207.05221 (2022). · arXiv
- Jun B Kim Paulo Albuquerque and Bart J Bronnenberg. 2011. Mapping online consumer search. Journal of Marketing Research 48 1 (2011) 13–27. · DOI
- Steve Lawrence. 2000. Context in Web Search. IEEE Data Engineering Bulletin 23 3 (2000) 25–32.
- 10.1145/3613904.3642428 · DOI
- Patrick Lewis Ethan Perez Aleksandra Piktus Fabio Petroni Vladimir Karpukhin Naman Goyal Heinrich Küttler Mike Lewis Wen-tau Yih Tim Rocktäschel et al. 2020. Retrieval-augmented generation for knowledge-intensive nlp tasks. Advances in Neural Information Processing Systems 33 (2020) 9459–9474.
- Stephanie Lin Jacob Hilton and Owain Evans. 2022. Teaching Models to Express Their Uncertainty in Words. Transactions on Machine Learning Research (2022). https://openreview.net/forum?id=8s8K2UZGTZ
- Jiqun Liu. 2021. Deconstructing search tasks in interactive information retrieval: A systematic review of task dimensions and predictors. Information Processing & Management 58 3 (2021) 102522. Publisher: Elsevier. · DOI
- Jia Liu and Olivier Toubia. 2018. A semantic approach for estimating consumer content preferences from online search queries. Marketing Science 37 6 (2018) 930–952. · DOI
- Jia Liu and Olivier Toubia. 2020. Search query formation by strategic consumers. Quantitative Marketing and Economics 18 (2020) 155–194. · DOI
- Jia Liu Olivier Toubia and Shawndra Hill. 2021. Content-based model of web search behavior: An application to TV show search. Management Science 67 10 (2021) 6378–6398. · DOI
- 10.1145/3411764.3445562 · DOI · Paper page
- 10.1145/2600428.2609508 · DOI
- 10.1145/2806416.2806488 · DOI
- OpenAI. 2023. GPT-4 Technical Report. arXiv preprint arXiv:https://arXiv.org/abs/2303.08774 (2023). · arXiv
- Samir Passi, Shipi Dhanorkar, and Mihaela Vorvoreanu. 2024. Appropriate reliance on Generative AI: Research synthesis. Technical Report MSR-TR-2024-7. Microsoft. https://www.microsoft.com/en-us/research/publication/appropriate-reliance-on-generative-ai-research-synthesis/
- Samir Passi and Mihaela Vorvoreanu. 2022. Overreliance on AI: Literature Review. Technical Report MSR-TR-2022-12. Microsoft. https://www.microsoft.com/en-us/research/publication/overreliance-on-ai-literature-review/
- Sida Peng Eirini Kalliamvakou Peter Cihon and Mert Demirer. 2023. The impact of ai on developer productivity: Evidence from github copilot. arXiv preprint arXiv:https://arXiv.org/abs/2302.06590 (2023). · arXiv
- Girish N Punj and Richard Staelin. 1983. A model of consumer information search behavior for new automobiles. Journal of Consumer Research 9 4 (1983) 366–380. · DOI
- Abigail Sellen and Eric Horvitz. 2024. The rise of the AI Co-Pilot: Lessons for design from aviation and beyond. Commun. ACM 67 7 (2024) 18–23. · DOI
- Kathryn Sharpe Wessling Joel Huber and Oded Netzer. 2017. MTurk character misrepresentation: Assessment and solutions. Journal of Consumer Research 44 1 (2017) 211–230. · DOI · Paper page
- 10.1145/331403.331405 · DOI
- StatCounter. 2023. Search engine market share worldwide. https://gs.statcounter.com/search-engine-market-share
- Emma van Inwegen, Zanele T Munyikwa, and John J Horton. 2023. Algorithmic writing assistance on jobseekers’ resumes increases hires. Technical Report. National Bureau of Economic Research.
- Helena Vasconcelos Gagan Bansal Adam Fourney Q Vera Liao and Jennifer Wortman Vaughan. 2024. Generation probabilities are not enough: Exploring the effectiveness of uncertainty highlighting in AI-powered code completions. TOCHI 2024: ACM Transactions on Computer-Human Interaction (2024).
- 10.18653/v1/2023.findings-acl.551 · DOI · Paper page
Cited by
Sorted by citations
Kaiser, C., Kaiser, J., Schallner, R., & Schneider, S. (2025) · Proceedings of the Extended Abstracts of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706599.3720123 · Cited by 11
Schott, K., Silva, K., Frommholz, I., Mayr, P., Kern, D., & Hienert, D. (2026) · arXiv · doi:10.48550/arxiv.2607.03421
Metadata
- Authors
- Spatharioti, S. E., Rothschild, D., Goldstein, D. G., & Hofman, J. M.
- Affiliations
- Microsoft Research, New York, New York, USA
- Microsoft Research, NYC, New York, USA
- Venue
- Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems
- Publisher
- ACM
- Published
- 2025-04-25
- Pages
- 1-15
- Cited by
- 36
- References
- 12
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
