Cited by 25Explainable Artificial Intelligence (XAI)Usability and User Interface DesignHuman-Automation Interaction and Safety

Assessing Demand for Transparency in Intelligent Systems Using Machine Learning

Open access
Vorm, Eric S.·2018·2018 Innovations in Intelligent Systems and Applications (INISTA)

Abstract

Intelligent systems offering decision support can lessen cognitive load and improve the efficiency of decision making in a variety of contexts. These systems assist users by evaluating multiple courses of action and recommending the right action at the right time. Modern intelligent systems using machine learning introduce new capabilities in decision support, but they can come at a cost. Machine learning models provide little explanation of their outputs or reasoning process, making it difficult to determine when it is appropriate to trust, or if not, what went wrong. In order to improve trust and ensure appropriate reliance on these systems, users must be afforded increased transparency, enabling an understanding of the systems reasoning, and an explanation of its predictions or classifications. Here we discuss the salient factors in designing transparent intelligent systems using machine learning, and present the results of a user-centered design study. We propose design guidelines derived from our study, and discuss next steps for designing for intelligent system transparency.

Integrity signals

Registry

Crossref · 10.1109/inista.2018.8466328

Cites

33 of 33 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

We couldn’t retrieve this paper’s full text, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Cites

Flagged first, then by citations

1 of the 34 references couldn’t be matched to a known paper.

Cited by

Sorted by citations

Authors
Vorm, E. S.
Affiliations
Indiana University Indianapolis (US)
Corresponding author
Vorm, Eric S.
Venue
2018 Innovations in Intelligent Systems and Applications (INISTA)
Publisher
IEEE
Published
2018-07
Pages
1-7
OpenAlex
W2892099700
Cited by
25
References
34
Citations by year
2025: 2 · 2024: 2 · 2023: 6 · 2022: 3 · 2021: 7 · 2020: 4 · 2019: 1
FWCI
1.22
Type
conference paper
Language
en
Open access
Licence
public-domain
Locations
  • International Conference on Innovations in Intelligent Systems and Applications (INISTA) (publishedVersion)
  • IUScholarWorks (Indiana University) (submittedVersion, public-domain)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
  • Physical Sciences › Computer Science › Human-Computer Interaction › Usability and User Interface Design
  • Social Sciences › Psychology › Social Psychology › Human-Automation Interaction and Safety
Keywords
transparency, intelligent systems, explainable AI, decision support, trust in AI, user-centered design, design guidelines, cognitive load, machine learning models

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review