The DOI matches its DataCite record; the references it cites are not checked yet.
Raising the Stakes: Assessing the Influence of Stakes on User Reliance Behavior in Human-AI Decision-Making
Open accessPreprint — published as 10.1145/3774935.3806657 · Paper page
Abstract
Human-AI collaboration is often proposed to improve high-stakes decision-making, yet the influence of increased stakes and imperfect AI on decision-making strategies is not fully understood. Studying such behavior in realistic settings is challenging, as application-grounded evaluations are costly, rely on experts, or lack meaningful consequences for decision errors. To address this, we introduce Blockies, a parametric dataset generator for visual diagnostic tasks, and conduct an empirical study examining how perceived stakes influence reliance calibration and behavior. Results show that raised stakes lead to longer deliberation, but less calibrated reliance, with participants increasingly deferring to incorrect AI advice as decision time increased. These findings highlight that increased effort under higher stakes does not necessarily improve reliance calibration and show the importance of accounting for stakes when evaluating human-AI decision-making.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“In high-stakes environments such as medicine or autonomous driving, users tend to display amplified trust responses -sometimes resulting in over-reliance or defensive under-reliance due to accountability concerns (Johnson, 2025).”
From Investigating appropriate reliance on AI-Based decision support systems: the role of expertise, trust, and self-confidence (2025) · cites this paper as Johnson, 2025
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Research contrasting low-and high-stakes contexts has shown that increasing consequence severity can amplify trust dynamics and sometimes foster over-or under-reliance due to accountability pressures (Johnson, 2025).”
From Investigating appropriate reliance on AI-Based decision support systems: the role of expertise, trust, and self-confidence (2025) · cites this paper as Johnson, 2025
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The publisher hasn’t deposited this paper’s reference list.
Research works printed but not linked: 15
- [3][3] Christopher J. Anderson. 2006. The Psychology of Doing Nothing: Forms of Decision Avoidance Result from Reason and Emotion. https://papers.ssrn.com/ abstract=895727 -
- [6][6] Zana Buçinca, Maja Barbara Malaya, and Krzysztof Z. Gajos. 2021. To Trust or to Think: Cognitive Forcing Functions Can Reduce Overreliance on AI in AIassisted Decision-making. Proc. ACM Hum.-Comput. Interact. 5, CSCW1 (April 2021), 188:1-188:21. doi:10.1145/3449287 - · doi:10.1145/3449287 · Paper page
- [7][7] Tomás Capretto, Camen Piho, Ravin Kumar, Jacob Westfall, Tal Yarkoni, and Osvaldo A Martin. 2022. Bambi: A Simple Interface for Fitting Bayesian Linear Models in Python. Journal of Statistical Software 103, 15 (2022), 1-29. doi:10. 18637/jss.v103.i15 -
- [8][8] Tirtha Chanda, Katja Hauser, Sarah Hobelsberger, Tabea-Clara Bucher, Carina Nogueira Garcia, Emmanouil Chousakos, Iva Crnaric, Jovana Majstorovic, Tanya Foreman, Sandra Peternel, Irem Ozdemir, Raymond L. Barnhill, Mar Llamas-Velasco, Gabriela Poch, Sören Korsing, Wiebke Sondermann, Markus V. Heppt, Michael Erdmann, Sebastian Haferkamp, and Matthias Goebeler. 2024. Dermatologist-like explainable AI enhances trust and confidence in diagnosing melanoma. Nature Communications 15, 1 (Jan. 2024), 524. doi:10.1038/s41467023-43095-4 Publisher: Nature Publishing Group. - · doi:10.1038/s41467023-43095-4
- [13][13] Arthur Gretton, Alex Smola, Jiayuan Huang, Marcel Schmittfull, Karsten Borgwardt, and Bernhard Scholkopf. 2009. Covariate Shift by Kernel Mean Matching. Dataset shift in machine learning 3, 4 (2009). -
- [14][14] Peter Hase and Mohit Bansal. 2020. Evaluating Explainable AI: Which Algorithmic Explanations Help Users Predict Model Behavior? doi:10.48550/arXiv.2005. 01831 arXiv:2005.01831 [cs]. - · arXiv · doi:10.48550/arxiv.2005
- [17][17] David S. Johnson, Olya Hakobyan, Jonas Paletschek, and Hanna Drimalla. 2024. Explainable AI for Audio and Visual Affective Computing: A Scoping Review. IEEE Transactions on Affective Computing (2024), 1-20. doi:10.1109/TAFFC.2024. 3505269 Conference Name: IEEE Transactions on Affective Computing. - · doi:10.1109/taffc.2024
- [18][18] Vivian Lai, Chacha Chen, Alison Smith-Renner, Q. Vera Liao, and Chenhao Tan. 2023. Towards a Science of Human-AI Decision Making: An Overview of Design Space in Empirical Human-Subject Studies. In Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency (FAccT '23) . Association for Computing Machinery, New York, NY, USA, 1369-1385. doi:10.1145/ 3593013.3594087 -
- [19][19] Thao Le, Tim Miller, Liz Sonenberg, and Ronal Singh. 2024. Towards the New XAI: A Hypothesis-Driven Approach to Decision Support Using Evidence. In ECAI 2024 . IOS Press, 850-857. doi:10.3233/FAIA240571 - · doi:10.3233/faia240571
- [21][21] Wolfgang Lutz, Anne-Katharina Deisenhofer, Julian Rubel, Björn Bennemann, Julia Giesemann, Kaitlyn Poster, and Brian Schwartz. 2022. Prospective evaluation of a clinical decision support system in psychological therapy. Journal of Consulting and Clinical Psychology 90, 1 (Jan. 2022), 90-106. doi:10.1037/ ccp0000642 -
- [22][22] Katelyn Xiaoying Mei, Rock Yuren Pang, Alex Lyford, Lucy Lu Wang, and Katharina Reinecke. 2026. Passing the Buck to AI: How Individuals' Decision-Making Patterns Affect Reliance on AI. doi:10.48550/arXiv.2505.01537 arXiv:2505.01537 [cs]. - · arXiv · doi:10.48550/arxiv.2505.01537
- [30][30] Santiago I. Sautua. 2017. Does uncertainty cause inertia in decision making? An experimental study of the role of regret aversion and indecisiveness. Journal of Economic Behavior & Organization 136 (April 2017), 1-14. doi:10.1016/j.jebo. 2017.02.003 - · doi:10.1016/j.jebo
- [32][32] Leon Sixt, Martin Schuessler, Oana-Iuliana Popescu, Philipp Weiß, and Tim Landgraf. 2022. Do Users Benefit From Interpretable Vision? A User Study, Baseline, And Dataset. In Proceedings of the International Conference on Learning Representations . https://openreview.net/forum?id=v6s3HVjPerv -
- [39][39] Giulia Vilone and Luca Longo. 2020. Explainable Artificial Intelligence: a Systematic Review. https://arxiv.org/abs/2006.00093v4 - · arXiv
- [41][41] Alexandra Zytek, Dongyu Liu, Rhema Vaithianathan, and Kalyan Veeramachaneni. 2022. Sibyl: Understanding and Addressing the Usability Challenges of Machine Learning In High-Stakes Decision Making. IEEE Transactions on Visualization and Computer Graphics 28, 1 (Jan. 2022), 1161-1171. doi:10.1109/TVCG. 2021.3114864 Conference Name: IEEE Transactions on Visualization and Computer Graphics. · doi:10.1109/tvcg
Cited by
Dogru, E. Ö., & Krämer, N. C. (2025) · Journal of Decision Systems · doi:10.1080/12460125.2025.2593251 · Cited by 15
Enhancing Trust in Human-AI Collaboration: A Conceptual Review of Operator-AI Teamwork
Hosawi, A., & Stone, R. (2025) · International Journal of Advanced Computer Science and Applications · doi:10.14569/ijacsa.2025.0160701 · Cited by 8
Beyond Black Boxes: AI Certification Labels Increase Adoption through Trust
de Almeida, F., Glaum, J., Hildred, K., & Scott, I. J. (2026) · Psychology and Marketing · doi:10.1002/mar.70185 · Cited by 3
Human-Centered and Contextual Assessment of Human-AI Decision-Making Interventions
Nasr, S., & Johnson, D. S. (2025) · Contextualizing Explanations · doi:10.64136/mznt1005
Self-citationCRiDiT: Instantiating a run-time testbed for trust calibration in AI-infused systemsPreprint
Ding, Y., Herbaut, N., & Salinesi, C. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2609.24833
Metadata
- Authors
- Johnson, D. S.
- Affiliations
- Bielefeld University (DE)
- Corresponding author
- Johnson, David S.
- ORCID
- Johnson, David S.
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2025
- Pages
- 429-434
- arXiv
- 2503.03529
- OpenAlex
- W4416085118
- Cited by
- 2
- References
- 0
- Citations by year
- 2026: 1 · 2025: 1
- FWCI
- 13.45
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- ACM Conference on User Modeling, Adaptation and Personalization (UMAP) (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Funders
- Universität Bielefeld
- Topics
- Social Sciences › Human Factors and Ergonomics › Human-Automation Interaction and Safety
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Social Sciences › Decision Sciences › General Decision Sciences › Decision-Making and Behavioral Economics
- Keywords
- human-AI collaboration, decision making under uncertainty, trust in AI, human-AI decision-making, deliberation time, dataset construction, high-stakes decision making
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Raising the Stakes: Assessing the Influence of Stakes on User Reliance Behavior in Human-AI Decision-Making. https://app.cochat.ai/papers/10.48550/arxiv.2503.03529
- The paper
- Johnson, D. S. (2025). Raising the Stakes: Assessing the Influence of Stakes on User Reliance Behavior in Human-AI Decision-Making. arXiv. https://doi.org/10.48550/arxiv.2503.03529
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.