CoChat

How Time Pressure in Different Phases of Decision-Making Influences Human-AI Collaboration

Open access
Cao, Shiye, Gomez, Catalina, Huang, Chien-Ming · 2023 · Proceedings of the ACM on Human-Computer Interaction

The DOI matches its Crossref record; 57 of the 64 references it cites were found in a registry or an index, and 7 are not checked yet.

Abstract

Human cognitive and decision-making abilities depreciate under pressure, motivating the emergence of artificial intelligence (AI) systems as decision support tools to assist people in performing tasks under stress. In this work, we study human decision-making behavior and task performance under time pressure---induced from limitedinitial observation time (time to perform the task before providing an initial response without AI input) andfinal decision time (time to weigh an AI's suggestion before reaching a collective human-AI team answer)---for spatial reasoning and count estimation tasks. Our results show that, while the impact of initial observation time on AI-assisted decision-making was dependent on task nature, participants were more likely to follow AI suggestions when they were provided with longer final decision time; moreover, although participants generally tended to adhere to their initial responses, they had more agency when they were more logically engaged in a task. Our results offer a nuanced understanding of human-AI collaboration under time pressure in different phases of the decision-making process.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1145/3610068

Cites

57 of 64 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 64 references

Research works printed but not linked: 22

  1. Carlos Arteta and Andrew Lempitsky, Victor andZisserman. 2016. Counting in the Wild. In European Conference on Computer Vision.
  2. RJM Bruls and RM Kwee. 2020. Workload for radiologists during on-call hours: dramatic increase in the past 15 years. Insights into imaging 11, 1 (2020), 1--7.
  3. Zoran Bursac, C Heath Gauss, David Keith Williams, and David W Hosmer. 2008. Purposeful selection of variables in logistic regression. Source code for biology and medicine 3, 1 (2008), 1--8.
  4. Constance de Margerie-Mellon and Guillaume Chassagnon. 2022. Artificial intelligence: A critical review of applications for lung nodule and lung cancer. Diagnostic and Interventional Imaging (2022).
  5. Ruth Fernandez, Itiel E Dror, and Claire Smith. 2011. Spatial abilities of expert clinical anatomists: Comparison of abilities between novices, intermediates, and experts in anatomy. Anatomical sciences education 4, 1 (2011), 1--8.
  6. Susanne Gaube, Harini Suresh, Martina Raue, Alexander Merritt, Seth J Berkowitz, Eva Lermer, Joseph F Coughlin, John V Guttag, Errol Colak, and Marzyeh Ghassemi. 2021. Do as AI say: susceptibility in deployment of clinical decision-aids. NPJ digital medicine 4, 1 (2021), 1--8.
  7. Maia Jacobs, Melanie F Pradier, Thomas H McCoy, Roy H Perlis, Finale Doshi-Velez, and Krzysztof Z Gajos. 2021. How machine-learning recommendations influence clinician treatment selections: the example of antidepressant selection. Translational psychiatry 11, 1 (2021), 1--9.
  8. Irving L. Janis and Leon Mann. 1977. Decision Making: A Psychological Analysis of Conflict, Choice, and Commitment. The ANNALS of the American Academy of Political and Social Science 488 (1977).
  9. Daniel Kahneman. 1973. Attention and effort. Prentice-Hall series in experimental psychology (1973).
  10. 10.1016/s0933--3657(01)00077-x · doi:10.1016/s0933--3657(01)00077-x
  11. John D Lee and Katrina A See. 2004. Trust in automation: Designing for appropriate reliance. Human factors 46, 1 (2004), 50--80.
  12. 10.1016/s0166--4328(01)00361--8 · doi:10.1016/s0166--4328(01)00361--8
  13. Stephen Rice, David Keller, Gayle Hunt, and David Trafimow. 2009. Automation Dependency Under Time Pressure. In 2009 International Symposium on Aviation Psychology. 611.
  14. 10.1038/s41467-020--17419--7 · doi:10.1038/s41467-020--17419--7
  15. Ericka Rovira, Kathleen McGarry, and Raja Parasuraman. 2007. Effects of imperfect automation on decision making in a simulated command and control task. Human factors 49, 1 (2007), 76--87.
  16. Nadine B Sarter and Beth Schroeder. 2001. Supporting decision making and action selection under time pressure and uncertainty: The case of in-flight icing. Human factors 43, 4 (2001), 573--583.
  17. Sonia Jawaid Shaikh and Ignacio F Cruz. 2022. AI in human teams: effects on technology use, members' interactions, and creative performance under time scarcity. AI & SOCIETY (2022), 1--14.
  18. 10.1037/0882--7974.21.3.545 · doi:10.1037/0882--7974.21.3.545
  19. Harini Suresh, Natalie Lao, and Ilaria Liccardi. 2020. Misplaced Trust: Measuring the Interference of Machine Learning in Human Decision-Making. In 12th ACM Conference on Web Science. 315--324.
  20. 10.1111/j.1467--9450.1987.tb00769.x · doi:10.1111/j.1467--9450.1987.tb00769.x
  21. Luc A Wauters, Simon A de Crombrugghe, Nadia Nour, and Erik Matthysen. 1995. Do female roe deer in good condition produce more sons than daughters. Behavioral ecology and sociobiology 37, 3 (1995), 189--193.
  22. Zhuoran Xiong, Xiao-Yang Liu, Shan Zhong, Hongyang Yang, and Anwar Walid. 2018. Practical deep reinforcement learning approach for stock trading. arXiv preprint arXiv:1811.07522 (2018). · arXiv

Cited by

Metadata

Authors
Cao, S., Gomez, C., & Huang, C.-M.
Affiliations
  • Johns Hopkins University (US)
  • Johns Hopkins University, Baltimore, MD, USA
Venue
Proceedings of the ACM on Human-Computer Interaction
Published
2023-09-28
Volume
7
Issue
CSCW2
Pages
1-26
ISSN
2573-0142 (ISSN-L)
OpenAlex
W4387331470
Cited by
48
References
55
Citations by year
2026: 24 · 2025: 19 · 2024: 5
FWCI
14.24
Type
journal article
Language
en
Open access
Locations
Proceedings of the ACM on Human-Computer Interaction (publishedVersion, cc-by)
Funders
NSF (1840088)
Topics
  • Social Sciences › Human Factors and Ergonomics › Human-Automation Interaction and Safety
  • Social Sciences › Decision Sciences › General Decision Sciences › Decision-Making and Behavioral Economics
  • Social Sciences › Psychology › Social Psychology › Social Robot Interaction and HRI
Keywords
time pressure, human-AI collaboration, decision-making, decision support system, spatial reasoning, observation time, human agency, cognitive performance under stress

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.