Do Humans Trust Advice More if it Comes from AI?: An Analysis of Human-AI Interactions
The DOI matches its Crossref record; 18 of the 18 references it cites were found in a registry or an index.
Abstract
In decision support applications of AI, the AI algorithm's output is framed as a suggestion to a human user. The user may ignore this advice or take it into consideration to modify their decision. With the increasing prevalence of such human-AI interactions, it is important to understand how users react to AI advice. In this paper, we recruited over 1100 crowdworkers to characterize how humans use AI suggestions relative to equivalent suggestions from a group of peer humans across several experimental settings. We find that participants' beliefs about how human versus AI performance on a given task affects whether they heed the advice. When participants do heed the advice, they use it similarly for human and AI suggestions. Based on these results, we propose a two-stage, "activation-integration" model for human behavior and use it to characterize the factors that affect human-AI interactions.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Maintaining proper self-confdence is critical for humans to establish appropriate trust in AI.”
From Who Should I Trust: AI or Myself? Leveraging Human and AI Correctness Likelihood to Promote Appropriate Trust in AI-Assisted Decision-Making (2023) · cites this paper as [96]
In the citing paper
“Evidence shows that people's confdence in themselves signifcantly afects whether they will take AI's advice [19,62,96].”
From Who Should I Trust: AI or Myself? Leveraging Human and AI Correctness Likelihood to Promote Appropriate Trust in AI-Assisted Decision-Making (2023) · cites this paper as [96]
In the citing paper
“Supporting this notion, studies have shown that individuals with more medical experience are often less susceptible to automation advice [ 58 ], whereas those with less experience are more prone to automation bias, leading to overreliance on AI [ 23 ].”
From Psychological Factors Influencing Appropriate Reliance on AI-enabled Clinical Decision Support Systems: Experimental Web-Based Study Among Dermatologists (2025) · cites this paper as [58]
In the citing paper
“Our research identified medical experience as a significant factor influencing self-reported trust and reliance behavior, which is consistent with previous research [ 58 - 60 ].”
From Psychological Factors Influencing Appropriate Reliance on AI-enabled Clinical Decision Support Systems: Experimental Web-Based Study Among Dermatologists (2025) · cites this paper as [58]
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Fischler, M. A., & Bolles, R. C. (1981) · Communications of the ACM · doi:10.1145/358669.358692 · Cited by 25,893
"Why Should I Trust You?": Explaining the Predictions of Any Classifier
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939778 · Cited by 17,555
Lee, M. K. (2018) · Big Data & Society · doi:10.1177/2053951718756684 · Cited by 1,338
Random effects structure for testing interactions in linear mixed-effects models
Barr, D. J. (2013) · Frontiers in Psychology · doi:10.3389/fpsyg.2013.00328 · Cited by 782
Mann, L., Burnett, P., Radford, M., & Ford, S. (1997) · Journal of Behavioral Decision Making · doi:10.1002/(sici)1099-0771(199703)10:1<1::aid-bdm242>3.0.co;2-x · Cited by 449
The Perceived Utility of Human and Automated Aids in a Visual Detection Task
Dzindolet, M. T., Pierce, L. G., Beck, H. P., & Dawe, L. A. (2002) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/0018720024494856 · Cited by 386
The relative influence of advice from human experts and statistical methods on forecast adjustments
Önkal, D., Goodwin, P., Thomson, M., Gönül, S., & Pollock, A. (2009) · Journal of Behavioral Decision Making · doi:10.1002/bdm.637 · Cited by 341
Understanding algorithm aversion: When is advice from automation discounted?
Prahl, A., & Van Swol, L. (2017) · Journal of Forecasting · doi:10.1002/for.2464 · Cited by 308 · cited 4×
“Previous work has attempted to understand the effects of peer vs. expert advice [18], human vs. algorithmic advice [6, 18, 21, 22], task difficulty [10], and advice confidence [26, 27] on human utilization of advice.”
BackgroundMethodInfluentialEffects of task difficulty on use of advice
Gino, F., & Moore, D. A. (2007) · Journal of Behavioral Decision Making · doi:10.1002/bdm.539 · Cited by 272
Precision and accuracy of judgmental estimation
Yaniv, I., & Foster, D. P. (1997) · Journal of Behavioral Decision Making · doi:10.1002/(sici)1099-0771(199703)10:1<21::aid-bdm243>3.0.co;2-g · Cited by 203
Madhavan, P., & Wiegmann, D. A. (2007) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/001872007x230154 · Cited by 157
Xie, Y., Chen, M., Kao, D., Gao, G., & Chen, X. '. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376807 · Cited by 141
Sah, S., Moore, D. A., & MacCoun, R. J. (2013) · Organizational Behavior and Human Decision Processes · doi:10.1016/j.obhdp.2013.02.001 · Cited by 132
Effects of distance between initial estimates and advice on advice utilization
Schultze, T., Rakotoarisoa, A.-F., & Stefan, S.-H. (2015) · Judgment and Decision Making · doi:10.1017/s1930297500003922 · Cited by 76
Artificial Intelligence in the Health care Space: How We Can Trust What We Cannot Know
Feldman, R., Aldana, E., & Stein, K. (2019) · bepress Legal Repository · Cited by 56
A Statistical Approach to Adult Census Income Level Prediction
Chakrabarty, N., & Biswas, S. (2018) · International Conference on Advances in Computing, Communication Control and Networking (ICAC3N) · doi:10.1109/icacccn.2018.8748528 · Cited by 49
Mesbah, N., Tauchert, C., & Buxmann, P. (2021) · Hawaii International Conference on System Sciences (HICSS) · doi:10.24251/hicss.2021.496 · Cited by 15
Research works printed but not linked: 11
- Douglas Bates , Martin Mächler , Ben Bolker , and Steve Walker . 2014. Fitting linear mixed-effects models using lme4. arXiv preprint arXiv:1406.5823 ( 2014 ). Douglas Bates, Martin Mächler, Ben Bolker, and Steve Walker. 2014. Fitting linear mixed-effects models using lme4. arXiv preprint arXiv:1406.5823 (2014). · arXiv
- Susanne Gaube , Harini Suresh , Martina Raue , Alexander Merritt , Seth J Berkowitz , Eva Lermer , Joseph F Coughlin , John V Guttag , Errol Colak , and Marzyeh Ghassemi . 2021. Do as AI say: susceptibility in deployment of clinical decision-aids. NPJ digital medicine , Vol. 4 , 1 ( 2021 ), 1--8. Susanne Gaube, Harini Suresh, Martina Raue, Alexander Merritt, Seth J Berkowitz, Eva Lermer, Joseph F Coughlin, John V Guttag, Errol Colak, and Marzyeh Ghassemi. 2021. Do as AI say: susceptibility in deployment of clinical decision-aids. NPJ digital medicine, Vol. 4, 1 (2021), 1--8.
- Alexa Hagerty and Igor Rubinov . 2019. Global AI ethics: a review of the social impacts and ethical implications of artificial intelligence. arXiv preprint arXiv:1907.07892 ( 2019 ). Alexa Hagerty and Igor Rubinov. 2019. Global AI ethics: a review of the social impacts and ethical implications of artificial intelligence. arXiv preprint arXiv:1907.07892 (2019). · arXiv
- Nigel Harvey and Ilan Fischer . 1997. Taking advice: Accepting help, improving judgment, and sharing responsibility. Organizational behavior and human decision processes , Vol. 70 , 2 ( 1997 ), 117--133. Nigel Harvey and Ilan Fischer. 1997. Taking advice: Accepting help, improving judgment, and sharing responsibility. Organizational behavior and human decision processes, Vol. 70, 2 (1997), 117--133.
- Nigel Harvey , Clare Harries , and Ilan Fischer . 2000. Using advice and assessing its quality. Organizational behavior and human decision processes , Vol. 81 , 2 ( 2000 ), 252--273. Nigel Harvey, Clare Harries, and Ilan Fischer. 2000. Using advice and assessing its quality. Organizational behavior and human decision processes, Vol. 81, 2 (2000), 252--273.
- Ekaterina Jussupow Izak Benbasat and Armin Heinzl. 2020. Why are we averse towards Algorithms? A comprehensive literature Review on Algorithm aversion.. In ECIS. Ekaterina Jussupow Izak Benbasat and Armin Heinzl. 2020. Why are we averse towards Algorithms? A comprehensive literature Review on Algorithm aversion.. In ECIS.
- Mikhail Khodak , Nikunj Saunshi , and Kiran Vodrahalli . 2017. A large self-annotated corpus for sarcasm. arXiv preprint arXiv:1704.05579 ( 2017 ). Mikhail Khodak, Nikunj Saunshi, and Kiran Vodrahalli. 2017. A large self-annotated corpus for sarcasm. arXiv preprint arXiv:1704.05579 (2017). · arXiv
- Tim Miller . 2019. Explanation in artificial intelligence: Insights from the social sciences. Artificial intelligence , Vol. 267 ( 2019 ), 1--38. Tim Miller. 2019. Explanation in artificial intelligence: Insights from the social sciences. Artificial intelligence, Vol. 267 (2019), 1--38.
- Veronica Rotemberg , Nicholas Kurtansky , Brigid Betz-Stablein , Liam Caffery , Emmanouil Chousakos , Noel Codella , Marc Combalia , Stephen Dusza , Pascale Guitera , David Gutman , et almbox . 2021 . A patient-centric dataset of images and metadata for identifying melanomas using clinical context. Scientific data, Vol. 8 , 1 (2021), 1--8. Veronica Rotemberg, Nicholas Kurtansky, Brigid Betz-Stablein, Liam Caffery, Emmanouil Chousakos, Noel Codella, Marc Combalia, Stephen Dusza, Pascale Guitera, David Gutman, et almbox. 2021. A patient-centric dataset of images and metadata for identifying melanomas using clinical context. Scientific data, Vol. 8, 1 (2021), 1--8.
- Christoph Tauchert , Neda Mesbah , et almbox . 2019 . Following the Robot? Investigating Users' Utilization of Advice from Robo-Advisors.. In ICIS. AIS. Christoph Tauchert, Neda Mesbah, et almbox. 2019. Following the Robot? Investigating Users' Utilization of Advice from Robo-Advisors.. In ICIS. AIS.
- Lyn M Van Swol , Jihyun Esther Paik, and Andrew Prahl . 2018 . Advice recipients: The psychology of advice utilization. (2018). Lyn M Van Swol, Jihyun Esther Paik, and Andrew Prahl. 2018. Advice recipients: The psychology of advice utilization. (2018).
Cited by
Swanson, K., Wu, E., Zhang, A., Alizadeh, A. A., & Zou, J. (2023) · Cell · doi:10.1016/j.cell.2023.01.035 · Cited by 597
“Also, prospective trials can capture human-AI interactions that occur after deployment.(120) Continuous performance monitoring of ML algorithms after approval and postmarket surveillance mechanisms are necessary to ensure that the purported clinical benefits of ML hold up under various distribution shifts.”
BackgroundSelf-citationTrust in AI: progress, challenges, and future directions
Afroogh, S., Akbari, A., Malone, E., Kargar, M., & Alambeigi, H. (2024) · Humanities and Social Sciences Communications · doi:10.1057/s41599-024-04044-8 · Cited by 507
Prather, J., Reeves, B. N., Denny, P., Becker, B. A., Leinonen, J., Luxton-Reilly, A., Powell, G., Finnie-Ansley, J., & Santos, E. A. (2024) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3617367 · Cited by 256
Blinded, randomized trial of sonographer versus AI cardiac function assessment
He, B., Kwan, A. C., Cho, J. H., Yuan, N., Pollick, C., Shiota, T., Ebinger, J., Bello, N. A., Wei, J., Josan, K., et al. (2023) · Nature · doi:10.1038/s41586-023-05947-3 · Cited by 256
Self-citationAI-chatbots on the services frontline addressing the challenges and opportunities of agency
Chong, T., Yu, T., Keeling, D. I., & de Ruyter, K. (2021) · Journal of Retailing and Consumer Services · doi:10.1016/j.jretconser.2021.102735 · Cited by 245
Metadata
- Authors
- Vodrahalli, K., Daneshjou, R., Gerstenberg, T., & Zou, J.
- Affiliations
- Stanford University (US)
- Venue
- Proceedings of the 2022 AAAI/ACM Conference on AI, Ethics, and Society
- Publisher
- ACM
- Published
- 2022-07-26
- Pages
- 763-777
- ISSN
- 3065-8365 (ISSN-L)
- OpenAlex
- W4288058227
- Cited by
- 63
- References
- 16
- Citations by year
- 2026: 14 · 2025: 19 · 2024: 21 · 2023: 8 · 2022: 1
- FWCI
- 23.59
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- Proceedings of the AAAI/ACM Conference on AI Ethics and Society (publishedVersion)
- Topics
- Social Sciences › Human Factors and Ergonomics › Human-Automation Interaction and Safety
- Social Sciences › Psychology › Social Psychology › Social Robot Interaction and HRI
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Keywords
- human-AI interaction, decision support, advice taking, peer advice, crowdsourced experiments
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.