Beyond Recommendations: From Backward to Forward AI Support of Pilots' Decision-Making Process
The DOI matches its Crossref record; 45 of the 45 references it cites were found in a registry or an index.
Abstract
AI is anticipated to enhance human decision-making in high-stakes domains like aviation, but adoption is often hindered by challenges such as inappropriate reliance and poor alignment with users' decision-making. Recent research suggests that a core underlying issue is the recommendation-centric design of many AI systems, i.e., they give end-to-end recommendations and ignore the rest of the decision-making process. Alternative support paradigms are rare, and it remains unclear how the few that do exist compare to recommendation-centric support. In this work, we aimed to empirically compare recommendation-centric support to an alternative paradigm, continuous support, in the context of diversions in aviation. We conducted a mixed-methods study with 32 professional pilots in a realistic setting. To ensure the quality of our study scenarios, we conducted a focus group with four additional pilots prior to the study. We found that continuous support can support pilots' decision-making in a forward direction, allowing them to think more beyond the limits of the system and make faster decisions when combined with recommendations, though the forward support can be disrupted. Participants' statements further suggest a shift in design goal away from providing recommendations, to supporting quick information gathering. Our results show ways to design more helpful and effective AI decision support that goes beyond end-to-end recommendations.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Findings from the experiment suggest that both strategies may have functioned as a type of process-oriented support [83], with patient information synthesized on the display as it was being presented in the vignette.”
From To Recommend or Not to Recommend: Designing and Evaluating AI-Enabled Decision Support for Time-Critical Medical Events (2025) · cites this paper as [83]
In the citing paper
“Process-oriented support proposes that recommendations may be provided at later points in events to confirm or challenge the initial decisions of users [83].”
From To Recommend or Not to Recommend: Designing and Evaluating AI-Enabled Decision Support for Time-Critical Medical Events (2025) · cites this paper as [83]
In the citing paper
“Unlike recommendation-driven strategies, information-driven strategies do not center on providing recommendations [27, 45, 83].”
From To Recommend or Not to Recommend: Designing and Evaluating AI-Enabled Decision Support for Time-Critical Medical Events (2025) · cites this paper as [83]
In the citing paper
“Although we developed these implications from our research on decision support for trauma resuscitation, they may also apply to other time-critical events, such as firefighting and aviation [80, 83].”
From To Recommend or Not to Recommend: Designing and Evaluating AI-Enabled Decision Support for Time-Critical Medical Events (2025) · cites this paper as [83]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Algorithm aversion: People erroneously avoid algorithms after seeing them err.
Dietvorst, B. J., Simmons, J. P., & Massey, C. (2015) · Journal of Experimental Psychology: General · doi:10.1037/xge0000033 · Cited by 3,232 · cited 1×
“…(human relies on AI even when it is disadvantageous to do so) [2, 6, 8, 24, 32, 39, 43, 47, 56, 66] and underreliance (human rejects AI even when it would be beneficial to rely on it) [12–14 In practice, AI support often turns out less useful to decision makers than imagined [4, 34, 71].”
BackgroundArtificial intelligence in healthcare
Yu, K.-H., Beam, A. L., & Kohane, I. S. (2018) · Nature Biomedical Engineering · doi:10.1038/s41551-018-0305-z · Cited by 2,817 · cited 1×
“AI is projected to improve human decision-making in various high-stakes domains, such as health-care [72], finance [11], or law enforcement [50].”
BackgroundTask-Dependent Algorithm Aversion
Castelo, N., Bos, M. W., & Lehmann, D. R. (2019) · Journal of Marketing Research · doi:10.1177/0022243719851788 · Cited by 1,500 · cited 1×
“…(human relies on AI even when it is disadvantageous to do so) [2, 6, 8, 24, 32, 39, 43, 47, 56, 66] and underreliance (human rejects AI even when it would be beneficial to rely on it) [12–14 In practice, AI support often turns out less useful to decision makers than imagined [4, 34, 71].”
BackgroundBuçinca, Z., Malaya, M. B., & Gajos, K. Z. (2021) · arXiv · doi:10.48550/arxiv.2102.09692 · Cited by 1,124
Designing Theory-Driven User-Centric Explainable AI
Wang, D., Yang, Q., Abdul, A., & Lim, B. Y. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300831 · Cited by 906
Zhang, Y., Liao, Q. V., & Bellamy, R. K. E. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372852 · Cited by 895
Does the Whole Exceed its Parts? The Effect of AI Explanations on Complementary Team Performance
Bansal, G., Wu, T., Zhou, J., Fok, R., Nushi, B., Kamar, E., Ribeiro, M. T., & Weld, D. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445717 · Cited by 757
Does the Whole Exceed its Parts? The Effect of AI Explanations on Complementary Team Performance
Bansal, G., Wu, T., Zhou, J., Fok, R., Nushi, B., Kamar, E., Ribeiro, M. T., & Weld, D. S. (2020) · arXiv · doi:10.48550/arxiv.2006.14779 · Cited by 754
Manipulating and Measuring Model Interpretability
Poursabzi-Sangdeh, F., Goldstein, D. G., Hofman, J. M., Wortman Vaughan, J. W., & Wallach, H. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445315 · Cited by 728 · cited 1×
“…of inappropriate reliance , including both overreliance (human relies on AI even when it is disadvantageous to do so) [2, 6, 8, 24, 32, 39, 43, 47, 56, 66] and underreliance (human rejects AI even when it would be beneficial to rely on it) [12–14 In practice, AI support often turns out less…”
BackgroundManipulating and Measuring Model Interpretability
Poursabzi-Sangdeh, F., Goldstein, D. G., Hofman, J. M., Vaughan, J. W., & Wallach, H. (2018) · arXiv · doi:10.48550/arxiv.1802.07810 · Cited by 711
Principles of Explanatory Debugging to Personalize Interactive Machine Learning
Kulesza, T., Burnett, M., Wong, W.-K., & Stumpf, S. (2015) · Proceedings of the 20th International Conference on Intelligent User Interfaces · doi:10.1145/2678025.2701399 · Cited by 576
Expanding Explainability: Towards Social Transparency in AI systems
Ehsan, U., Liao, Q. V., Muller, M., Riedl, M. O., & Weisz, J. D. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445188 · Cited by 538 · cited 1×
“Instead of explaining the AI model, explanations can provide information that is of natural interest in a decision, such as domain-specific information [41, 69], or the socio-organizational context of a decision [18].”
BackgroundThe Role of Explanations on Trust and Reliance in Clinical Decision Support Systems
Bussone, A., Stumpf, S., & O'Sullivan, D. (2015) · 2015 International Conference on Healthcare Informatics · doi:10.1109/ichi.2015.26 · Cited by 441 · cited 4×
“This indicates a potentially significant gap between what is studied in large-scale controlled experiments and what is actually required by experts in real applications.”
InfluentialHuman-Centered Tools for Coping with Imperfect Algorithms During Medical Decision-Making
Cai, C. J., Reif, E., Hegde, N., Hipp, J., Kim, B., Smilkov, D., Wattenberg, M., Viegas, F., Corrado, G. S., Stumpe, M. C., et al. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300234 · Cited by 405
Explanations Can Reduce Overreliance on AI Systems During Decision-Making
Vasconcelos, H., Jörke, M., Grunde-McLaughlin, M., Gerstenberg, T., Bernstein, M. S., & Krishna, R. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3579605 · Cited by 402
The Principles and Limits of Algorithm-in-the-Loop Decision Making
Green, B., & Chen, Y. (2019) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3359152 · Cited by 392 · cited 1×
“This is the result of inappropriate reliance , including both overreliance (human relies on AI even when it is disadvantageous to do so) [2, 6, 8, 24, 32, 39, 43, 47, 56, 66] and underreliance (human rejects AI even when it would be beneficial to rely on it) [12–14 In practice, AI support often…”
BackgroundLai, V., & Tan, C. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287590 · Cited by 377
A Review of User Interface Design for Interactive Machine Learning
Dudley, J. J., & Kristensson, P. O. (2018) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3185517 · Cited by 343
Wang, X., & Yin, M. (2021) · 26th International Conference on Intelligent User Interfaces · doi:10.1145/3397481.3450650 · Cited by 320
Understanding algorithm aversion: When is advice from automation discounted?
Prahl, A., & Van Swol, L. (2017) · Journal of Forecasting · doi:10.1002/for.2464 · Cited by 308
Dix, A. (2007) · Electronic Workshops in Computing · doi:10.14236/ewic/hci2007.53 · Cited by 304
Jacobs, M., Pradier, M. F., McCoy, T. H., Perlis, R. H., Doshi-Velez, F., & Gajos, K. Z. (2021) · Translational Psychiatry · doi:10.1038/s41398-021-01224-x · Cited by 247 · cited 4×
“This is not only an issue for pilots as in our study, but also in other domains, such as in healthcare [21, 32, 65].”
BackgroundResultInfluentialYang, Q., Steinfeld, A., & Zimmerman, J. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300468 · Cited by 247
Deciding Fast and Slow: The Role of Cognitive Biases in AI-assisted Decision-making
Rastogi, C., Zhang, Y., Wei, D., Varshney, K. R., Dhurandhar, A., & Tomsett, R. (2022) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3512930 · Cited by 216
McGuirl, J. M., & Sarter, N. B. (2006) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/001872006779166334 · Cited by 212 · cited 2×
“Communicating model confidence appears more promising, as it has repeatedly been shown to improve appropriate reliance [2, 44, 48, 74]; but this relies on well-calibrated confidence scores, which are often difficult to achieve—models can be wrong with high confidence.”
BackgroundAppropriate Reliance on AI Advice: Conceptualization and the Effect of Explanations
Schemmer, M., Kuehl, N., Benz, C., Bartos, A., & Satzger, G. (2023) · Proceedings of the 28th International Conference on Intelligent User Interfaces · doi:10.1145/3581641.3584066 · Cited by 208
Lai, V., Chen, C., Smith-Renner, A., Liao, Q. V., & Tan, C. (2023) · 2023 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3593013.3594087 · Cited by 204
Jacobs, M., He, J., F. Pradier, M., Lam, B., Ahn, A. C., McCoy, T. H., Perlis, R. H., Doshi-Velez, F., & Gajos, K. Z. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445385 · Cited by 177
Miller, T. (2023) · 2023 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3593013.3594001 · Cited by 172
The Impact of Placebic Explanations on Trust in Intelligent Systems
Eiband, M., Buschek, D., Kremer, A., & Hussmann, H. (2019) · Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290607.3312787 · Cited by 152 · cited 1×
“Results have been mixed so far, as especially explanations are prone to induce blind trust [2, 19] and hence overreliance, even among domain experts [6, 32].”
BackgroundKawakami, A., Sivaraman, V., Cheng, H.-F., Stapleton, L., Cheng, Y., Qing, D., Perer, A., Wu, Z. S., Zhu, H., & Holstein, K. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3517439 · Cited by 143
Sivaraman, V., Bukowski, L. A., Levin, J., Kahn, J. M., & Perer, A. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581075 · Cited by 141
Do People Engage Cognitively with AI? Impact of AI Assistance on Incidental Learning
Gajos, K. Z., & Mamykina, L. (2022) · 27th International Conference on Intelligent User Interfaces · doi:10.1145/3490099.3511138 · Cited by 127
Smith-Renner, A., Fan, R., Birchfield, M., Wu, T., Boyd-Graber, J., Weld, D. S., & Findlater, L. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376624 · Cited by 112
Liu, H., Lai, V., & Tan, C. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3479552 · Cited by 111
Yang, Q., Zimmerman, J., Steinfeld, A., Carey, L., & Antaki, J. F. (2016) · Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems · doi:10.1145/2858036.2858373 · Cited by 105
Zhang, S., Yu, J., Xu, X., Yin, C., Lu, Y., Yao, B., Tory, M., Padilla, L. M., Caterino, J., Zhang, P., et al. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642343 · Cited by 99
Rechkemmer, A., & Yin, M. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3501967 · Cited by 97
Prabhudesai, S., Yang, L., Asthana, S., Huan, X., Liao, Q. V., & Banovic, N. (2023) · Proceedings of the 28th International Conference on Intelligent User Interfaces · doi:10.1145/3581641.3584033 · Cited by 82 · cited 1×
“Communicating model confidence appears more promising, as it has repeatedly been shown to improve appropriate reliance [2, 44, 48, 74]; but this relies on well-calibrated confidence scores, which are often difficult to achieve—models can be wrong with high confidence.”
BackgroundFok, R., & Weld, D. S. (2024) · AI Magazine · doi:10.1002/aaai.12182 · Cited by 81
A Slow Algorithm Improves Users' Assessments of the Algorithm's Accuracy
Park, J. S., Barber, R., Kirlik, A., & Karahalios, K. (2019) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3359204 · Cited by 65 · cited 3×
“However, with cognitive forcing interventions, people get no support while making their independent decisions, which reduces the supportive value of the AI [8, 21, 46].”
BackgroundInfluentialThe need to move away from agential-AI: Empirical investigations, useful concepts and open issues
Cabitza, F., Campagner, A., & Simone, C. (2021) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2021.102696 · Cited by 55
Fok, R., & Weld, D. S. (2023) · arXiv · doi:10.48550/arxiv.2305.07722 · Cited by 54
Bach, A. K. P., Nørgaard, T. M., Brok, J. C., & van Berkel, N. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581513 · Cited by 47
Calibrating Human-AI Collaboration: Impact of Risk, Ambiguity and Transparency on Algorithmic Bias
Schmidt, P., & Biessmann, F. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-57321-8_24 · Cited by 26 · cited 2×
“…of inappropriate reliance , including both overreliance (human relies on AI even when it is disadvantageous to do so) [2, 6, 8, 24, 32, 39, 43, 47, 56, 66] and underreliance (human rejects AI even when it would be beneficial to rely on it) [12–14 In practice, AI support often turns out less…”
Research works printed but not linked: 34
- 10.1007/978--3-031-02217--3_5 · doi:10.1007/978--3-031-02217--3_5
- 10.1111/1559-8918.2018.01208 · DOI
- 10.1145/3544548.3581251 · DOI · Paper page
- Zana Buçinca, Alexandra Chouldechova, Jennifer Wortman Vaughan, and Krzysztof Z. Gajos. 2022. Beyond end predictions: stop putting machine learning first and design human-centered AI for decision support. In Virtual Workshop on Human-Centered AI Workshop at NeurIPS (HCAI @ NeurIPS '22). Virtual Event, USA, 1--4.
- 10.1145/3449287 · doi:10.1145/3449287 · Paper page
- 10.1145/3502289 · DOI · Paper page
- 10.1145/3544548.3581253 · DOI
- 10.1145/3564040 · doi:10.1145/3564040
- 10.1145/3531146.3533193 · DOI · Paper page
- 10.1145/3544548.3580694 · DOI · Paper page
- 10.1108/00022660710780650 · DOI
- 10.1145/3610067 · DOI · Paper page
- 10.1207/s15327108ijap0901_2 · DOI · Paper page
- 10.1145/3581641.3584052 · DOI · Paper page
- Hans-Jürgen Hörmann. 1994. FOR-DEC - A prescriptive model for aeronautical decision making. In Proceedings of the 21st Conference of the European Association for Aviation Psychology (EAAP). Avebury Aviation, Dublin, Ireland, 17--23.
- 10.1145/3357236.3395436 · DOI · Paper page
- 10.1145/3532106.3533556 · DOI · Paper page
- Sean Koon. 2022. A human-capabilities orientation for human-AI interaction design. In Virtual Workshop on Human-Centered AI Workshop at NeurIPS (HCAI @ NeurIPS '22). Virtual Event, USA, 1--5.
- 10.3357/AMHP.5720.2021 · doi:10.3357/amhp.5720.2021
- 10.48550/arXiv.2302.01241 · DOI
- 10.1145/3397481.3450681 · DOI
- 10.1109/MSEC.2019.2925649 · DOI · Paper page
- 10.1007/978--3-030--21451--7_22 · doi:10.1007/978--3-030--21451--7_22
- 10.1007/978--3-030--57321--8_24 · doi:10.1007/978--3-030--57321--8_24
- 10.1518/001872007X312469 · DOI · Paper page
- 10.1109/TTS.2020.2992669 · DOI · Paper page
- 10.1109/3468.568744 · DOI · Paper page
- 10.1145/3422156 · DOI · Paper page
- 10.1159/000511930 · DOI · Paper page
- 10.1007/978--3--642--50329-0_11 · doi:10.1007/978--3--642--50329-0_11
- 10.54941/ahfe1003844 · DOI
- 10.1145/3544548.3581393 · DOI · Paper page
- 10.1145/3464385.3464696 · DOI
- 10.1145/3581641.3584056 · DOI
Cited by
AIdeation: Designing a Human-AI Collaborative Ideation System for Concept Designers
Wang, W.-F., Lu, C.-T., Ponsa i Campanyà, N., Chen, B.-Y., & Chen, M. Y. (2025) · Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3714148 · Cited by 50
“In aviation, the AI Support System for Pilots’ Decision-Making Process [37] highlights the importance of transparent feedback loops, enabling pilots to understand AI recommendations.”
Mastrianni, A., Kim, M. S., Sullivan, T. M., Sippel, G. J., Burd, R. S., Gajos, K. Z., & Sarcevic, A. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3757512 · Cited by 6
Metadata
- Authors
- Zhang, Z. T., Feger, S. S., Dullenkopf, L., Liao, R., Süsslin, L., Liu, Y., & Butz, A.
- Affiliations
- fortiss GmbH & LMU Munich, Munich, Germany
- LMU Munich & TH Rosenheim, Munich, Germany
- Airbus Defence and Space GmbH, Manching, Germany
- LMU Munich, Munich, Germany
- TU Wien, Vienna, Austria
- Research Institute of the Free State of Bavaria, fortiss GmbH, Munich, Germany
- ORCID
- Venue
- Proceedings of the ACM on Human-Computer Interaction
- Published
- 2024-11-07
- Volume
- 8
- Issue
- CSCW2
- Pages
- 1-32
- ISSN
- 2573-0142
- Cited by
- 16
- References
- 45
- Language
- en
- Funders
- German Federal Ministry for Economic Affairs and Climate Action (LuFo VI-1 program, project KIEZ4-0)
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.