How Time Pressure in Different Phases of Decision-Making Influences Human-AI Collaboration
Open accessThe DOI matches its Crossref record; 57 of the 57 references it cites were found in a registry or an index.
Abstract
Human cognitive and decision-making abilities depreciate under pressure, motivating the emergence of artificial intelligence (AI) systems as decision support tools to assist people in performing tasks under stress. In this work, we study human decision-making behavior and task performance under time pressure---induced from limitedinitial observation time (time to perform the task before providing an initial response without AI input) andfinal decision time (time to weigh an AI's suggestion before reaching a collective human-AI team answer)---for spatial reasoning and count estimation tasks. Our results show that, while the impact of initial observation time on AI-assisted decision-making was dependent on task nature, participants were more likely to follow AI suggestions when they were provided with longer final decision time; moreover, although participants generally tended to adhere to their initial responses, they had more agency when they were more logically engaged in a task. Our results offer a nuanced understanding of human-AI collaboration under time pressure in different phases of the decision-making process.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Automatic and effortful processes in memory.
Hasher, L., & Zacks, R. T. (1979) · Journal of Experimental Psychology: General · doi:10.1037/0096-3445.108.3.356 · Cited by 2,812
Guidelines for Human-AI Interaction
Amershi, S., Weld, D., Vorvoreanu, M., Fourney, A., Nushi, B., Collisson, P., Suh, J., Iqbal, S., Bennett, P. N., Inkpen, K., et al. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300233 · Cited by 2,439 · cited 1×
“Effective human-AI teaming is challenging to design and achieve [2, 77]; to enable successful human-AI collaboration, previous research has investigated how a range of factors—including model capabilities, user backgrounds, and task contexts—may shape people’s performance with and trust in an AI…”
BackgroundBuçinca, Z., Malaya, M. B., & Gajos, K. Z. (2021) · arXiv · doi:10.48550/arxiv.2102.09692 · Cited by 1,124
Zhang, Y., Liao, Q. V., & Bellamy, R. K. E. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372852 · Cited by 895
Derksen, S., & Keselman, H. J. (1992) · British Journal of Mathematical and Statistical Psychology · doi:10.1111/j.2044-8317.1992.tb00992.x · Cited by 771
Does the Whole Exceed its Parts? The Effect of AI Explanations on Complementary Team Performance
Bansal, G., Wu, T., Zhou, J., Fok, R., Nushi, B., Kamar, E., Ribeiro, M. T., & Weld, D. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445717 · Cited by 757 · cited 2×
“For example, prior works have explored how information elements, such as explanations of model outputs [6, 11], performance and confidence values [78, 81], and details about training data and model architecture [12, 66], or user involvement in joint decision making tasks [20] may influence human-AI…”
BackgroundDoes the Whole Exceed its Parts? The Effect of AI Explanations on Complementary Team Performance
Bansal, G., Wu, T., Zhou, J., Fok, R., Nushi, B., Kamar, E., Ribeiro, M. T., & Weld, D. S. (2020) · arXiv · doi:10.48550/arxiv.2006.14779 · Cited by 754
The harassed decision maker: Time pressures, distractions, and the use of evidence.
Wright, P. (1974) · Journal of Applied Psychology · doi:10.1037/h0037186 · Cited by 751
Re-examining Whether, Why, and How Human-AI Interaction Is Uniquely Difficult to Design
Yang, Q., Steinfeld, A., Rosé, C., & Zimmerman, J. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376301 · Cited by 622 · cited 1×
“Effective human-AI teaming is challenging to design and achieve [2, 77]; to enable successful human-AI collaboration, previous research has investigated how a range of factors—including model capabilities, user backgrounds, and task contexts—may shape people’s performance with and trust in an AI…”
BackgroundPruessner, J. C., Dedovic, K., Khalili-Mahani, N., Engert, V., Pruessner, M., Buss, C., Renwick, R., Dagher, A., Meaney, M. J., & Lupien, S. (2008) · Biological Psychiatry · doi:10.1016/j.biopsych.2007.04.041 · Cited by 593
Research works printed but not linked: 25
- Carlos Arteta and Andrew Lempitsky, Victor andZisserman. 2016. Counting in the Wild. In European Conference on Computer Vision.
- RJM Bruls and RM Kwee. 2020. Workload for radiologists during on-call hours: dramatic increase in the past 15 years. Insights into imaging 11, 1 (2020), 1--7.
- 10.1145/3449287 · doi:10.1145/3449287 · Paper page
- Zoran Bursac, C Heath Gauss, David Keith Williams, and David W Hosmer. 2008. Purposeful selection of variables in logistic regression. Source code for biology and medicine 3, 1 (2008), 1--8.
- Constance de Margerie-Mellon and Guillaume Chassagnon. 2022. Artificial intelligence: A critical review of applications for lung nodule and lung cancer. Diagnostic and Interventional Imaging (2022).
- Ruth Fernandez, Itiel E Dror, and Claire Smith. 2011. Spatial abilities of expert clinical anatomists: Comparison of abilities between novices, intermediates, and experts in anatomy. Anatomical sciences education 4, 1 (2011), 1--8.
- Susanne Gaube, Harini Suresh, Martina Raue, Alexander Merritt, Seth J Berkowitz, Eva Lermer, Joseph F Coughlin, John V Guttag, Errol Colak, and Marzyeh Ghassemi. 2021. Do as AI say: susceptibility in deployment of clinical decision-aids. NPJ digital medicine 4, 1 (2021), 1--8.
- 10.1093/oxfordhb/9780198795872.013.22 · DOI
- Maia Jacobs, Melanie F Pradier, Thomas H McCoy, Roy H Perlis, Finale Doshi-Velez, and Krzysztof Z Gajos. 2021. How machine-learning recommendations influence clinician treatment selections: the example of antidepressant selection. Translational psychiatry 11, 1 (2021), 1--9.
- Irving L. Janis and Leon Mann. 1977. Decision Making: A Psychological Analysis of Conflict, Choice, and Commitment. The ANNALS of the American Academy of Political and Social Science 488 (1977).
- Daniel Kahneman. 1973. Attention and effort. Prentice-Hall series in experimental psychology (1973).
- 10.1016/S0933--3657(01)00077-X · doi:10.1016/s0933--3657(01)00077-x
- John D Lee and Katrina A See. 2004. Trust in automation: Designing for appropriate reliance. Human factors 46, 1 (2004), 50--80.
- 10.1016/S0166--4328(01)00361--8 · doi:10.1016/s0166--4328(01)00361--8
- Stephen Rice, David Keller, Gayle Hunt, and David Trafimow. 2009. Automation Dependency Under Time Pressure. In 2009 International Symposium on Aviation Psychology. 611.
- 10.1038/s41467-020--17419--7 · doi:10.1038/s41467-020--17419--7
- Ericka Rovira, Kathleen McGarry, and Raja Parasuraman. 2007. Effects of imperfect automation on decision making in a simulated command and control task. Human factors 49, 1 (2007), 76--87.
- Nadine B Sarter and Beth Schroeder. 2001. Supporting decision making and action selection under time pressure and uncertainty: The case of in-flight icing. Human factors 43, 4 (2001), 573--583.
- Sonia Jawaid Shaikh and Ignacio F Cruz. 2022. AI in human teams: effects on technology use, members' interactions, and creative performance under time scarcity. AI & SOCIETY (2022), 1--14.
- 10.1037/0882--7974.21.3.545 · doi:10.1037/0882--7974.21.3.545
- Robert S Stawski et al. 2006. Stress-related cognitive interference predicts cognitive function in old age. Psychology and aging 21 3 (2006) 535--544. https://doi.org/10.1037/0882--7974.21.3.535 · doi:10.1037/0882--7974.21.3.535
- Harini Suresh, Natalie Lao, and Ilaria Liccardi. 2020. Misplaced Trust: Measuring the Interference of Machine Learning in Human Decision-Making. In 12th ACM Conference on Web Science. 315--324.
- 10.1111/j.1467--9450.1987.tb00769.x · doi:10.1111/j.1467--9450.1987.tb00769.x
- Luc A Wauters, Simon A de Crombrugghe, Nadia Nour, and Erik Matthysen. 1995. Do female roe deer in good condition produce more sons than daughters. Behavioral ecology and sociobiology 37, 3 (1995), 189--193.
- Zhuoran Xiong, Xiao-Yang Liu, Shan Zhong, Hongyang Yang, and Anwar Walid. 2018. Practical deep reinforcement learning approach for stock trading. arXiv preprint arXiv:1811.07522 (2018). · arXiv
Cited by
Exploring automation bias in human–AI collaboration: a review and implications for explainable AI
Romeo, G., & Conti, D. (2026) · AI & SOCIETY · doi:10.1007/s00146-025-02422-7 · Cited by 209
“Overreliance is further reinforced by high workload (Vered et al. 2023; Zhang et al. 2024; Alami et al. 2025), time pressure (Cao et al. 2023), and task complexity (Bond et al. 2018; Kupfer et al. 2023; Cecil et al. 2024).”
InfluentialAccuracy-Time Tradeoffs in AI-Assisted Decision Making under Time Pressure
Swaroop, S., Buçinca, Z., Gajos, K. Z., & Doshi-Velez, F. (2024) · Proceedings of the 29th International Conference on Intelligent User Interfaces · doi:10.1145/3640543.3645206 · Cited by 49
“Cao et al. [13] looked at the impact of time pressure on the AI-after assistance type, finding that reliance on AI recommendations can sometimes increase or decrease, depending on (i) exactly when time pressure is added during the decision-making process, and (ii) the type of task.”
Backgroundde Jong, S., Paananen, V., Tag, B., & van Berkel, N. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3710946 · Cited by 32
Co-Writing with AI, on Human Terms: Aligning Research with User Demands Across the Writing Process
Reza, M., Thomas-Mitchell, J., Dushniku, P., Laundry, N., Williams, J. J., & Kuzminykh, A. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3757566 · Cited by 37
Toward a science of human–AI teaming for decision making: a complementarity framework
Gonzalez, C., Donahue, K., Goldstein, D. G., Heidari, H., Jalali, M. S., Schelble, B., Singh, A., & Woolley, A. W. (2026) · PNAS Nexus · doi:10.1093/pnasnexus/pgag030 · Cited by 33
Metadata
- Authors
- Cao, S., Gomez, C., & Huang, C.-M.
- Affiliations
- Johns Hopkins University (US)
- Johns Hopkins University, Baltimore, MD, USA
- Venue
- Proceedings of the ACM on Human-Computer Interaction
- Published
- 2023-09-28
- Volume
- 7
- Issue
- CSCW2
- Pages
- 1-26
- ISSN
- 2573-0142 (ISSN-L)
- OpenAlex
- W4387331470
- Cited by
- 48
- References
- 57
- Citations by year
- 2026: 24 · 2025: 19 · 2024: 5
- FWCI
- 14.24
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Locations
- Proceedings of the ACM on Human-Computer Interaction (publishedVersion, cc-by)
- Funders
- NSF (1840088)
- Topics
- Social Sciences › Human Factors and Ergonomics › Human-Automation Interaction and Safety
- Social Sciences › Decision Sciences › General Decision Sciences › Decision-Making and Behavioral Economics
- Social Sciences › Psychology › Social Psychology › Social Robot Interaction and HRI
- Keywords
- time pressure, human-AI collaboration, decision-making, decision support system, spatial reasoning, observation time, human agency, cognitive performance under stress
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: How Time Pressure in Different Phases of Decision-Making Influences Human-AI Collaboration. https://app.cochat.ai/papers/10.1145/3610068
- The paper
- Cao, S., Gomez, C., & Huang, C.-M. (2023). How Time Pressure in Different Phases of Decision-Making Influences Human-AI Collaboration. Proceedings of the ACM on Human-Computer Interaction. https://doi.org/10.1145/3610068
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.