The DOI matches its Crossref record; 6 of the 6 references it cites were found in a registry or an index.
Effects of transparency on pilot trust and agreement in the autonomous constrained flight planner
Abstract
We performed a human-in-the-loop study to explore the role of transparency in engendering trust and reliance within highly automated systems. Specifically, we examined how transparency impacts trust in and reliance upon the Autonomous Constrained Flight Planner (ACFP), a critical automated system being developed as part of NASA's Reduced Crew Operations (RCO) Concept. The ACFP is designed to provide an enhanced ground operator, termed a super dispatcher, with recommended diversions for aircraft when their primary destinations are unavailable. In the current study, 12 commercial transport rated pilots who played the role of super dispatchers were given six time-pressured “all land” scenarios where they needed to use the ACFP to determine diversions for multiple aircraft. Two factors were manipulated. The primary factor was level of transparency. In low transparency scenarios the pilots were given a recommended airport and runway, plus basic information about the weather conditions, the aircraft types, and the airport and runway characteristics at that and other airports. In moderate transparency scenarios the pilots were also given a risk evaluation for the recommended airport, and for the other airports if they requested it. In the high transparency scenario additional information including the reasoning for the risk evaluations was made available to the pilots. The secondary factor was level of risk, either high or low. For high-risk aircraft, all potential diversions were rated as highly risky, with the ACFP giving the best option for a bad situation. For low-risk aircraft the ACFP found only low-risk options for the pilot. Both subjective and objective measures were collected, including rated trust, whether the pilots checked the validity of the automation recommendation, and whether the pilots eventually flew to the recommended diversion airport. Key results show that: 1) Pilots' trust increased with higher levels of transparency, 2) Pilots were more likely to verify ACFP's recommendations with low levels of transparency and when risk was high, 3) Pilots were more likely to explore other options from the ACFP in low transparency conditions and when risk was high, and 4) Pilots' decision to accept or reject ACFP's recommendations increased as a function of the transparency in the explanation. The finding that higher levels of transparency was coupled with higher levels of trust, a lower need to verify other options, and higher levels of agreement with ACFP recommendations, confirms the importance of transparency in aiding reliance on automated recommendations. Additional analyses of qualitative data gathered from subjects through surveys and during debriefing interviews also provided the basis for new design recommendations for the ACFP.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“high transparency may create a level of complacency that leads to less vigilance in overseeing or questioning the autonomous agent’s work (Sadler et al., 2016).”
From Human–Autonomy Teaming: A Review and Analysis of the Empirical Literature (2022) · cites this paper as Sadler et al., 2016
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Trust in Automation: Designing for Appropriate Reliance
Lee, J. D., & See, K. A. (2004) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/hfes.46.1.50_30392 · Cited by 6,605 · cited 5×
“A. Information Seeking The Baseline conditions showed no trends associated with Risk, with pilots verifying 61%-65% of the time, and exploring 73%-74% of the time.”
BackgroundInfluentialHumans and automation: Use, misuse, disuse, abuse
Parasuraman, R., & Riley, V. (1997) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/001872097778543886 · Cited by 4,554
A model for types and levels of human interaction with automation
Parasuraman, R., Sheridan, T. B., & Wickens, C. D. (2000) · IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans · doi:10.1109/3468.844354 · Cited by 4,195 · cited 1×
“Using this transparency decomposition, systems can be designed to encourage human-automation teaming and a fitting division of labor [9] between the system and user.”
MethodTrust in Automation: Integrating Empirical Evidence on Factors That Influence Trust
Hoff, K. A., & Bashir, M. (2015) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720814547570 · Cited by 2,817
The effects of transparency on trust in and acceptance of a content-based art recommender
Cramer, H., Evers, V., Ramlal, S., van Someren, M., Rutledge, L., Stash, N., Aroyo, L., & Wielinga, B. (2008) · User Modeling and User-Adapted Interaction · doi:10.1007/s11257-008-9051-3 · Cited by 448
Intelligent agent transparency in human–agent teaming for multi-UxV management
Mercado, J. E., Rupp, M. A., Chen, J. Y. C., Barnes, M. J., Barber, D., & Procci, K. (2016) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720815621206 · Cited by 403
Research works printed but not linked: 17
- 10.21236/ADA600351 · DOI · Paper page
- 10.1518/001872095779049543 · DOI · Paper page
- 10.1109/TSMCC.2010.2056682 · DOI · Paper page
- 10.1016/S0020-7373(87)80013-5 · DOI · Paper page
- 10.1177/0018720812443825 · DOI · Paper page
- 10.1177/1064804615611272 · DOI · Paper page
- lyons, 2016, Shaping trust through transparent design: theoretical and experimental guidelines, Conf Applied Human Factors and Ergonomics
- 10.1007/978-3-319-20373-7_40 · doi:10.1007/978-3-319-20373-7_40
- 10.1177/0018720811427034 · DOI
- 10.1518/hfes.46.1.50.30392 · doi:10.1518/hfes.46.1.50.30392 · Paper page
- lyons, 2013, 2013 Assoc for the Advancement of Artificial Intell Spring Symp, 48
- 10.1109/THMS.2013.2293535 · DOI · Paper page
- 0, Nat. Aeronautics and Space Admin. (2015, December 1), Unmanned Aircraft System (UAS) Traffic Management (UTM)
- 10.2514/6.2016-1002 · DOI
- meuleau, 2009, Emergency landing planner for damaged aircraft, Proc 21st Innovative Applicat of Artificial Intell Conf, 114
- 10.1109/SMC.2014.6974381 · DOI
- 10.1016/j.promfg.2015.07.846 · doi:10.1016/j.promfg.2015.07.846
Cited by
Human–Autonomy Teaming: A Review and Analysis of the Empirical Literature
O’Neill, T., McNeese, N., Barron, A., & Schelble, B. (2022) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720820960865 · Cited by 695
Zhang, R., Duan, W., Flathmann, C., McNeese, N., Freeman, G., & Williams, A. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3610072 · Cited by 134
Calibrating Reliance on Automated Advice: Transparency and Trust Calibration Feedback
Tatasciore, M., & Loft, S. (2025) · International Journal of Human–Computer Interaction · doi:10.1080/10447318.2025.2487861 · Cited by 20
Metadata
- Authors
- Sadler, G., Battiste, H., Ho, N., Hoffmann, L., Johnson, W., Shively, R., Lyons, J., & Smith, D.
- Venue
- 2016 IEEE/AIAA 35th Digital Avionics Systems Conference (DASC)
- Publisher
- IEEE
- Published
- 2016-09
- Pages
- 1-9
- Cited by
- 34
- References
- 6
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.