The DOI matches its Crossref record; 130 of the 130 references it cites were found in a registry or an index; none of the 84 sources its claims cite is checked for support yet.
Human-AI collaboration is not very collaborative yet: a taxonomy of interaction patterns in AI-assisted decision making from a systematic review
Open accessAbstract
Leveraging Artificial Intelligence (AI) in decision support systems has disproportionately focused on technological advancements, often overlooking the alignment between algorithmic outputs and human expectations. A human-centered perspective attempts to alleviate this concern by designing AI solutions for seamless integration with existing processes. Determining what information AI should provide to aid humans is vital, a concept underscored by explainable AI's efforts to justify AI predictions. However, how the information is presented, e.g., the sequence of recommendations and solicitation of interpretations, is equally crucial as complex interactions may emerge between humans and AI. While empirical studies have evaluated human-AI dynamics across domains, a common vocabulary for human-AI interaction protocols is lacking. To promote more deliberate consideration of interaction designs, we introduce a taxonomy of interaction patterns that delineate various modes of human-AI interactivity. We summarize the results of a systematic review of AI-assisted decision making literature and identify trends and opportunities in existing interactions across application domains from 105 articles. We find that current interactions are dominated by simplistic collaboration paradigms, leading to little support for truly interactive functionality. Our taxonomy offers a tool to understand interactivity with AI in decision-making and foster interaction designs for achieving clear communication, trustworthiness, and collaboration.
Overview
CoChat Summary
TL;DR
This systematic review of 105 empirical studies proposes a taxonomy of interaction patterns for AI-assisted decision making. It finds that current human-AI interactions are dominated by simple AI-first and AI-follow assistance, with limited support for more interactive collaboration.
Key findings
- Across 105 articles and 131 interaction sequences, AI-first assistance was most common (n = 67), followed by AI-follow assistance (n = 28); AI solutions were accompanied by additional information in 81% and 68% of those patterns, respectively.
- More interactive patterns were less frequent: Request-driven AI assistance (n = 25; 14 for direct solution, 13 for support information), Secondary assistance (n = 16), User-guided interactive adjustments (n = 9), Delegation (n = 9), and AI-guided dialogic user engagement (n = 6, of which 5 were conversational).
- Domains included healthcare (26/108), generic tasks (20/108), finance/business (15/108), and law (6/108); most evaluations used non-expert users (60/108), and less than 15% used artificial tasks. AI systems were simulated/Wizard of Oz (39/105), deep learning-based (34/105), or shallow models (35/105).
- The taxonomy comprises seven interaction patterns—AI-first assistance, AI-follow assistance, Secondary assistance, Request-driven AI assistance, AI-guided dialogic user engagement, User-guided interactive adjustments, and Delegation—plus an “Others” category.
Methods
- Systematic review of empirical human-AI decision-making studies from 2013 to June 2023 across ACM Digital Library, IEEE Xplore, Compendex, Scopus, and PubMed; the initial search returned 3,770 papers, 358 duplicates were removed, 3,412 were screened, and 105 articles were included.
- Inclusion required complete decision-making tasks, an implemented interface for human-AI interaction, screen-based/virtual/non-embodied setups, and empirical user studies; robotics, gaming, short papers (<8 pages), non-peer-reviewed works, and reviews/surveys were excluded.
- Data extraction coded context, AI system, and interaction building blocks (action-output pairs); interaction sequences were grouped iteratively into the taxonomy, and pattern frequency was quantified via automated search plus manual inspection.
Limitations
- Stated: search terminology may have excluded relevant work if interaction design was not explicitly mentioned; publication bias is possible; including only one healthcare-focused database alongside general databases may limit observed domain frequencies; analysis was restricted to screen-based interfaces; strict complete-decision-task criteria were used; some papers lacked sufficient interface detail for full recreation; and further use-case evaluations are needed to assess the taxonomy’s utility.
- Evident from the text: because interaction patterns are not mutually exclusive and the taxonomy was iteratively refined by the authors, category assignments may involve subjective interpretation.
- Evident from the text: the corpus ends in June 2023, so newer interaction patterns or studies published after that date are not represented.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures
Verification
Claims the paper makes
0 Valid · 0 Not valid · 45 Not checked · 126 Own results and interpretations
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Human-AI interactions in AI-advised decision makings in the healthcare domain mostly adhered to the AI-first assistance pattern (n = 14), followed by Secondary assistance (n = 7).”
§ Human-AI collaboration is not very collaborative yet: a taxonomy of interaction patterns in AI-assisted decision making from a systematic review > Domain Decision making task Total tasks
plainn = 7- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We identified one interaction that supported AI-guided dialogic user engagement, three interactions in the User-guided interactive adjustments, and one in which delegation was an option.”
§ Human-AI collaboration is not very collaborative yet: a taxonomy of interaction patterns in AI-assisted decision making from a systematic review > Domain Decision making task Total tasks
plainn = 3- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We present seven categories of interaction patterns that we have identified in our corpus, illustrated in Figure 1.”
§ Human-AI collaboration is not very collaborative yet: a taxonomy of interaction patterns in AI-assisted decision making from a systematic review > Results
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A check of a cited source is scheduled, or we don’t hold its full text yet; the claim is checked once it runs.
In the paper
“For instance, manipulations of the inputs result in new AI's outcome computations for exploratory purposes (Liu et al., 2021;Gu et al., 2023;Nguyen et al., 2018;Zytek et al., 2022;Suresh et al., 2022).”
§ Human-AI collaboration is not very collaborative yet: a taxonomy of interaction patterns in AI-assisted decision making from a systematic review > . Identification of interaction patterns in AI-assisted decision making studies
What the sources say
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Even though we identified multiple decision making tasks in applications that require a specialized population, where recruiting large numbers of participants is often challenging, the majority of human-AI interactions have been evaluated with non-expert uses (60/108).”
§ Human-AI collaboration is not very collaborative yet: a taxonomy of interaction patterns in AI-assisted decision making from a systematic review > Domain Decision making task Total tasks
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
81 more claims
Create a free account to see all 171 claims and to verify the 77 references still open.
- Probably not a substantial claim. 73
- It may state two separate claims as one. 26
- The quote only partly states this claim. 2
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
“The type of AI systems behind the interactions with users in the studies that covered these decision-making tasks were distributed among three categories: simulated models or Wizard of Oz experiments (39/105), deep learning-based models (34/105), and shallow models (35/105).”
§ Human-AI collaboration is not very collaborative yet: a taxonomy of interaction patterns in AI-assisted decision making from a systematic review > Domain Decision making task Total tasks
Reported 105 · recomputed 108
Citations
Cites
A model for types and levels of human interaction with automation
Parasuraman, R., Sheridan, T. B., & Wickens, C. D. (2000) · IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans · doi:10.1109/3468.844354 · Cited by 4,195 · cited 1×
“We searched for relevant articles from five databases that cover Human-Computer Interaction (HCI) studies and related disciplines and selected 105 to conduct a detailed coding and analysis of the sequences of interactions that exist between humans and AI.”
Buçinca, Z., Malaya, M. B., & Gajos, K. Z. (2021) · arXiv · doi:10.48550/arxiv.2102.09692 · Cited by 1,124 · cited 3×
“…the decision making problem, we observe cases ( n = 10) in which users are given the ability to control when they want to receive the AI assistance via a request (Khadpe et al., 2020; Mackeprang et al., 2019; Baudel et al., 2021; Molina and Sundar, 2022; Gomez et al., 2023; Buçinca et al., 2021).”
Power to the People: The Role of Humans in Interactive Machine Learning
Amershi, S., Cakmak, M., Knox, W. B., & Kulesza, T. (2014) · AI Magazine · doi:10.1609/aimag.v35i4.2513 · Cited by 1,084
Toward a Deeper Understanding of the Role of Interaction in Information Visualization
Yi, J. S., Kang, Y. A., Stasko, J., & Jacko, J. A. (2007) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2007.70515 · Cited by 1,031
Human-computer collaboration for skin cancer recognition
Tschandl, P., Rinner, C., Apalla, Z., Argenziano, G., Codella, N., Halpern, A., Janda, M., Lallas, A., Longo, C., Malvehy, J., et al. (2020) · Nature Medicine · doi:10.1038/s41591-020-0942-0 · Cited by 1,024
Zhang, Y., Liao, Q. V., & Bellamy, R. K. E. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372852 · Cited by 895
Does the Whole Exceed its Parts? The Effect of AI Explanations on Complementary Team Performance
Bansal, G., Wu, T., Zhou, J., Fok, R., Nushi, B., Kamar, E., Ribeiro, M. T., & Weld, D. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445717 · Cited by 757 · cited 2×
“…al., 2022; Riveiro and Thill, 2021, 2022; Lai et al., 2020; Bansal et al., 2021); question answering (Feng and Boyd-Graber, 2022; Silva et al., 2023; Bansal et al., 2021); speech classification (Tutul et al., 2021; Zhang and Lim, 2022) 20 Labeling Text labeling (Bernard et al., 2018; Schrills and…”
Does the Whole Exceed its Parts? The Effect of AI Explanations on Complementary Team Performance
Bansal, G., Wu, T., Zhou, J., Fok, R., Nushi, B., Kamar, E., Ribeiro, M. T., & Weld, D. S. (2020) · arXiv · doi:10.48550/arxiv.2006.14779 · Cited by 754
Fügener, A., Grahl, J., Gupta, A., & Ketter, W. (2022) · Information Systems Research · doi:10.1287/isre.2021.1079 · Cited by 478
Beyond Accuracy: The Role of Mental Models in Human-AI Team Performance
Bansal, G., Nushi, B., Kamar, E., Lasecki, W. S., Weld, D. S., & Horvitz, E. (2019) · Proceedings of the AAAI Conference on Human Computation and Crowdsourcing · doi:10.1609/hcomp.v7i1.5285 · Cited by 466
Research works printed but not linked: 1
- David, 2022, “Interaction templates: a data-driven approach for authoring robot programs,”, PLATEAU: 12th Annual Workshop at the Intersection of PL and HCI
Cited by
Items 201–211 of 211
Liu, Y., Li, J., Siau, K. L., & Xu, D. (. (n.d.) · Journal of the Association for Information Systems
Strategies for Students' Interaction with Generative AI as a Tool for Psychological Support
Savelyeva, E. S. (2026) · Almanac Institute of special education · doi:10.66435/almanac.ise.63.08
Mapping the Human-AI Collaboration in Strategic Decision-Making: A Bibliometric Review
Bidari, S., Sharma, L. R., & Molla, N. (2026) · NPRC Journal of Multidisciplinary Research. · doi:10.3126/nprcjmr.v3i10.100699
Bekar, F. (2026) · Bucak İşletme Fakültesi Dergisi · doi:10.38057/bifd.1945318
A Literature-Based Interaction Pattern Catalogue For Systems Engineering Assistants
Tülp, B., Koldewey, C., & Dumitrescu, R. (2026) · doi:10.35199/norddesign2026.53
Fischer, S. W., Schraffenberger, H., van Hooff, M. L., Reijnen, C., Thill, S., & Haselager, P. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2610.40224
Kim, D., & Jeong, Y. (2026) · Behavioral Sciences · doi:10.3390/bs16112050
EFL learners’ epistemic agency in AI-supported digital storytelling: a rhizomatic analysis
Wang, W.-H., & Yu, B. (2026) · Computer Assisted Language Learning · doi:10.1080/09588221.2026.2756909
When AI Influence Helps or Hurts: Guarantees for Ranking-Based Human–AI Aggregation
Karge, J., Ferguson, R., Grimaldi, D., Haldimann, J. P., Madzime, R., & Meyer, T. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-43514-9_6
Human–AI Cognitive Division of Labor: A Structured Narrative Review and Integrative Framework
Chen, C., Li, G., Yuan, S., Xu, Q., & Cai, C. (2026) · Applied Sciences · doi:10.3390/app162110970
Singh, S., Hindriks, K. V., Heylen, D., & Baraka, K. (2026) · International Conference on Human-Agent Interaction (HAI) · doi:10.1145/3841580.3841607
Metadata
- Authors
- Gomez, C., Cho, S. M., Ke, S., Huang, C.-M., & Unberath, M.
- Affiliations
- Johns Hopkins University (US)
- Corresponding author
- Unberath, Mathias
- Venue
- Frontiers in Computer Science
- Publisher
- Frontiers Media SA
- Published
- 2025-01-06
- Volume
- 6
- ISSN
- 2624-9898 (ISSN-L)
- OpenAlex
- W4406083100
- Cited by
- 209
- References
- 129
- Citations by year
- 2026: 160 · 2025: 46
- FWCI
- 20.79
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Frontiers in Computer Science (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion, cc-by-nc-nd)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- arXiv (Cornell University) (cc-by-nc-nd)
- Funders
- Johns Hopkins University
- Topics
- Social Sciences › Human Factors and Ergonomics › Human-Automation Interaction and Safety
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Physical Sciences › Computer Science › Human-Computer Interaction › Usability and User Interface Design
- Keywords
- human-AI collaboration, AI-assisted decision-making, interaction design, decision support system, explainable AI, trust in AI, human-centered AI, interaction protocols
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-11-14, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-11-14 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.
