Bringing Transparency Design into Practice
The DOI matches its Crossref record; 39 of the 40 references it cites were found in a registry or an index.
Abstract
Intelligent systems, which are on their way to becoming mainstream in everyday products, make recommendations and decisions for users based on complex computations. Researchers and policy makers increasingly raise concerns regarding the lack of transparency and comprehensibility of these computations from the user perspective. Our aim is to advance existing UI guidelines for more transparency in complex real-world design scenarios involving multiple stakeholders. To this end, we contribute a stage-based participatory process for designing transparent interfaces incorporating perspectives of users, designers, and providers, which we developed and validated with a commercial intelligent fitness coach. With our work, we hope to provide guidance to practitioners and to pave the way for a pragmatic approach to transparency in intelligent systems.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Transparency of the inner workings of the system [6,23] and interactivity to probe and manipulate the system [16] have been demonstrated to help people interpret and interact with intelligent systems to achieve their goals.”
From Memory Sandbox: Transparent and Interactive Memory Management for Conversational Agents (2023) · cites this paper as [6]
Citations
Cites
Using thematic analysis in psychology
Braun, V., & Clarke, V. (2006) · Qualitative Research in Psychology · doi:10.1191/1478088706qp063oa · Cited by 194,414
A BEHAVIORAL MODEL OF RATIONAL CHOICE
Simon, H. A. (1955) · The Quarterly Journal of Economics · doi:10.2307/1884852 · Cited by 15,877 · cited 1×
“Based on our learnings in this case study, we hypothesize that users might follow the principle of satisficing [42] also in the context of transparency: Transparency satisfices users if it allows them to build mental models good enough to predict and explain the observed system behavior, even though the given explanat…”
BackgroundSUS: A 'Quick and Dirty' Usability Scale
(1996) · Usability Evaluation In Industry · doi:10.1201/9781498710411-35 · Cited by 8,989
Johnson-Laird, P. N. (1983) · Cited by 5,190
Telerobotics, Automation, and Human Supervisory Control
Sheridan, T. B. (1992) · Medical Entomology and Zoology · Cited by 1,949
Craik, K. J. W. (1943) · Cited by 1,845
Explaining collaborative filtering recommendations
Herlocker, J. L., Konstan, J. A., & Riedl, J. (2000) · Proceedings of the 2000 ACM conference on Computer supported cooperative work · doi:10.1145/358916.358995 · Cited by 1,731 · cited 7×
“[15] explored a variety of explanation types and presentation formats; their results suggest that simpler graphs are more appealing to users.”
BackgroundInfluentialSome observations on mental models
Norman, D. A. (1983) · Medical Entomology and Zoology · Cited by 1,549
The role of trust in automation reliance
Dzindolet, M. T., Peterson, S. A., Pomranky, R. A., Pierce, L. G., & Beck, H. P. (2003) · International Journal of Human-Computer Studies · doi:10.1016/s1071-5819(03)00038-7 · Cited by 1,276 · cited 1×
“Moreover, trust in the system and its predictions is diminished in opaque systems [11, 29].”
BackgroundUser experience evaluation methods: current state and development needs
Vermeeren, A. P. O. S., Law, E. L.-C., Roto, V., Obrist, M., Hoonhout, J., & Väänänen-Vainio-Mattila, K. (2010) · Proceedings of the 6th Nordic Conference on Human-Computer Interaction: Extending Boundaries · doi:10.1145/1868914.1868973 · Cited by 592 · cited 1×
“These suggestions are based on literature review and on the experiences made during our project, and are meant to facilitate the application of our process, but should not be taken as the only methods possible – other methods from the plethora of participatory design research methods [38, 46] might be equally appropri…”
MethodPrinciples of Explanatory Debugging to Personalize Interactive Machine Learning
Kulesza, T., Burnett, M., Wong, W.-K., & Stumpf, S. (2015) · Proceedings of the 20th International Conference on Intelligent User Interfaces · doi:10.1145/2678025.2701399 · Cited by 576 · cited 11×
“, [21, 23, 27, 35]), there is no work on how to integrate transparency into existing applications yet.”
BackgroundInfluentialWhy and why not explanations improve the intelligibility of context-aware intelligent systems
Lim, B. Y., Dey, A. K., & Avrahami, D. (2009) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1518701.1519023 · Cited by 552 · cited 3×
“− Workshops − Interviews − Online surveys [28] − Hypothetical scenarios [25] − Semi-structured interviews − Data collection in problem-solving tasks [32] − Drawing tasks [8, 18] − In-depth interviews − Focus groups − Hypothetical scenarios [25] − Card sorting [47] − Focus groups, workshops and brainstorming sessions −…”
BackgroundMethodExplanations From Intelligent Systems: Theoretical Foundations and Implications for Practice1
Gregor, S., & Benbasat, I. (1999) · MIS Quarterly · doi:10.2307/249487 · Cited by 535 · cited 11×
“Similarly, Schaffer et al. [40] as well as Gregor and Benbasat [13] highlight the importance of considering the user group or individual differences between users when thinking about what to explain : Explanations turned out to be more useful when tailored to the interest of a specific user group, and may even reduce u…”
BackgroundResultInfluentialThe role of transparency in recommender systems
Sinha, R., & Swearingen, K. (2002) · CHI '02 Extended Abstracts on Human Factors in Computing Systems · doi:10.1145/506443.506619 · Cited by 477 · cited 1×
“Another concept closely linked to transparency is trust in an intelligent system [41], which is of crucial importance for safety-related systems, for example, airplanes [29], but does also influence user satisfaction with systems such as recommenders [43].”
BackgroundIntelligibility and Accountability: Human Considerations in Context-Aware Systems
Bellotti, V., & Edwards, K. (2001) · Human–Computer Interaction · doi:10.1207/s15327051hci16234_05 · Cited by 471 · cited 1×
“Bellotti and Edwards [1] call for appropriate abstraction from the system model when informing users about the underlying calculations.”
BackgroundThe effects of transparency on trust in and acceptance of a content-based art recommender
Cramer, H., Evers, V., Ramlal, S., van Someren, M., Rutledge, L., Stash, N., Aroyo, L., & Wielinga, B. (2008) · User Modeling and User-Adapted Interaction · doi:10.1007/s11257-008-9051-3 · Cited by 448 · cited 6×
“While prior work argues that both communications from the systems to the user and from the user to the system are important, for example for correcting system mistakes [7], the focus in our case, the Freeletics project, was on transparency of the system only.”
BackgroundInfluentialToo much, too little, or just right? Ways explanations impact end users' mental models
Kulesza, T., Stumpf, S., Burnett, M., Yang, S., Kwan, I., & Wong, W.-K. (2013) · 2013 IEEE Symposium on Visual Languages and Human Centric Computing · doi:10.1109/vlhcc.2013.6645235 · Cited by 388 · cited 3×
“Moreover, mental models as a system-indepentent concept have been used in prior work as a way to approach the design of transparent intelligent systems in different contexts [22, 23, 24].”
BackgroundMethodEvaluating recommender systems from the user’s perspective: survey of the state of the art
Pu, P., Chen, L., & Hu, R. (2012) · User Modeling and User-Adapted Interaction · doi:10.1007/s11257-011-9115-7 · Cited by 377 · cited 1×
“A notable exception is the widely adopted use of labels in recommender systems [37].”
BackgroundTrust building with explanation interfaces
Pu, P., & Chen, L. (2006) · Proceedings of the 11th international conference on Intelligent user interfaces · doi:10.1145/1111449.1111475 · Cited by 300 · cited 10×
“, [21, 23, 27, 35]), there is no work on how to integrate transparency into existing applications yet.”
BackgroundInfluentialTrust-inspiring explanation interfaces for recommender systems
Pu, P., & Chen, L. (2007) · Knowledge-Based Systems · doi:10.1016/j.knosys.2007.04.004 · Cited by 248
Assessing demand for intelligibility in context-aware applications
Lim, B. Y., & Dey, A. K. (2009) · Proceedings of the 11th international conference on Ubiquitous computing · doi:10.1145/1620545.1620576 · Cited by 246 · cited 10×
“These methods were selected because they allowed us to investigate users’ mental models as they emerge through long-term product use while the algorithm processes their own data as input (in contrast to hypothetical scenarios [25]) and to generate in-depth qualitative data (in contrast to online surveys [28]).”
BackgroundMethodInfluentialTell me more?: the effects of mental model soundness on personalizing an intelligent agent
Kulesza, T., Stumpf, S., Burnett, M., & Kwan, I. (2012) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/2207676.2207678 · Cited by 225 · cited 9×
“, explaining how the system works, has been shown to improve users’ mental models of that system [22, 23].”
BackgroundMethodInfluentialExplaining How a Deep Neural Network Trained with End-to-End Learning Steers a Car
Bojarski, M., Yeres, P., Choromanska, A., Choromański, K., Firner, B., Jackel, L. D., & Müller, U. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1704.07911 · Cited by 223 · cited 1×
“However, this view would imply that undoubtedly useful machine learning algorithms, in particular deep learning algorithms, such as used in computer vision and self-driving cars, were not to be applied in practice as researchers are still working on ways to make their decisions explainable [2].”
BackgroundGetting the right design and the design right
Tohidi, M., Buxton, W., Baecker, R., & Sellen, A. (2006) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1124772.1124960 · Cited by 211 · cited 1×
“For the same reason, if multiple prototypes have been developed (as in our project), it is sensible to recruit different user groups for each prototype to investigate the effect of the different design solutions on users’ mental models (in contrast to the general recommendation to show users more than one prototype to…”
BackgroundCard sorting: current practices and beyond
Wood, J. R., & Wood, L. E. (2008) · Journal of Usability Studies archive · Cited by 182
Toolkit to support intelligibility in context-aware applications
Lim, B. Y., & Dey, A. K. (2010) · Proceedings of the 12th ACM international conference on Ubiquitous computing · doi:10.1145/1864349.1864353 · Cited by 143 · cited 4×
“Lim and Dey [25] have argued that implementing all intelligibility types is “excessive and may even be detrimental”.”
BackgroundInfluentialHow it works: a field study of non-technical users interacting with an intelligent system
Tullio, J., Dey, A. K., Chalecki, J., & Fogarty, J. (2007) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1240624.1240630 · Cited by 137 · cited 4×
“’s [45] observation that users are able to form a lay, but surprisingly accurate mental model of machine learning concepts.”
BackgroundInfluentialExplanatory Debugging: Supporting End-User Debugging of Machine-Learned Programs
Kulesza, T., Stumpf, S., Burnett, M., Wong, W.-K., Riche, Y., Moore, T., Oberst, I., Shinsel, A., & McIntosh, K. (2010) · 2010 IEEE Symposium on Visual Languages and Human-Centric Computing · doi:10.1109/vlhcc.2010.15 · Cited by 93 · cited 5×
“, explaining how the system works, has been shown to improve users’ mental models of that system [22, 23].”
BackgroundMethodInfluentialIntelligent aids, mental models, and the theory of machines
Moray, N. P. (1987) · International Journal of Man-Machine Studies · doi:10.1016/s0020-7373(87)80020-2 · Cited by 84 · cited 2×
“However, most mental models are simplified representations of the actual system workings [32, 45] – which is sufficient if it allows users to comprise the majority of observed system behavior [30].”
BackgroundShaping Trust Through Transparent Design: Theoretical and Experimental Guidelines
Lyons, J. B., Sadler, G. G., Koltai, K., Battiste, H., Ho, N., Hoffmann, L. C., Smith, D. E., Johnson, W. W., & Shively, R. J. (2016) · Advances in intelligent systems and computing · doi:10.1007/978-3-319-41959-6_11 · Cited by 78 · cited 3×
“Moreover, trust in the system and its predictions is diminished in opaque systems [11, 29].”
BackgroundKay, J., & Kummerfeld, B. (2012) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/2395123.2395129 · Cited by 71 · cited 1×
“Explaining how a system works might enable users to correct system decisions through feedback (as in [21, 23]), might improve understanding of the system recommendations and thus foster efficient or effective use [35, 36], prevent mis- and disuse of a system through increased trust [39] or protect users’ privacy [19]…”
BackgroundUser Preferences for Hybrid Explanations
Kouki, P., Schaffer, J., Pujara, J., O'Donovan, J., & Getoor, L. (2017) · Proceedings of the Eleventh ACM Conference on Recommender Systems · doi:10.1145/3109859.3109915 · Cited by 71 · cited 1×
“[20] investigated user preferences for visualizations of explanations in recommenders, and found that Venn diagrams performed best in comparison to other visual interfaces, but were restricted by the number of information items presented.”
BackgroundEmpirical Studies on Software Notices to Inform Policy Makers and Usability Designers
Großklags, J., & Good, N. (2007) · Lecture notes in computer science · doi:10.1007/978-3-540-77366-5_31 · Cited by 49
Design of an intelligible mobile context-aware application
Lim, B. Y., & Dey, A. K. (2011) · Proceedings of the 13th International Conference on Human Computer Interaction with Mobile Devices and Services · doi:10.1145/2037373.2037399 · Cited by 45 · cited 2×
“However, when the authors applied the toolkit in a later study, they concluded that it was difficult to translate the automatically generated explanations into textual descriptions [27].”
BackgroundMethodBest practices for capturing context in user experience studies in the wild
Roto, V., Väätäjä, H., Pyykko, S. J., & Väänänen, K. (2011) · International Academic Mindtrek Conference (Mindtrek) · doi:10.1145/2181037.2181054 · Cited by 42 · cited 1×
“These suggestions are based on literature review and on the experiences made during our project, and are meant to facilitate the application of our process, but should not be taken as the only methods possible – other methods from the plethora of participatory design research methods [38, 46] might be equally appropri…”
MethodGetting the Message?: A Study of Explanation Interfaces for Microblog Data Analysis
Schaffer, J., Giridhar, P., Jones, D., Höllerer, T., Abdelzaher, T., & O'Donovan, J. (2015) · Proceedings of the 20th International Conference on Intelligent User Interfaces · doi:10.1145/2678025.2701406 · Cited by 39 · cited 5×
“This is in agreement with calls for context-specific design solutions for explanations [13, 15, 35, 40] using a participatory approach [15, 25].”
BackgroundResultInfluentialDinet, J., & Kitajima, M. (2011) · Conference on l'Interaction Homme-Machine (IHM) · doi:10.1145/2044354.2044358 · Cited by 32
The New Imbroglio – Living with Machine Algorithms
Hildebrandt, M. (2016) · VUBIR (Vrije Universiteit Brussel) · doi:10.25969/mediarep/13395 · Cited by 11
Jackson, K. F., Prasov, Z., Vincent, E. C., & Jones, E. M. (2016) · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · doi:10.1177/1541931213601390 · Cited by 7 · cited 1×
“Trust was not assessed during our project, but future work might investigate how our process and improved mental models affect trust, using frameworks such as [17].”
Research works printed but not linked: 2
- ACM U.S. Public Policy Council and ACM Europe Policy Committee . 2017 . Statement on Algorithmic Transparency and Accountability . (2017). https://www.acm.org/binaries/content/assets/public-policy/2017_joint_statement_algorithms.pdf ACM U.S. Public Policy Council and ACM Europe Policy Committee. 2017. Statement on Algorithmic Transparency and Accountability. (2017). https://www.acm.org/binaries/content/assets/public-policy/2017_joint_statement_algorithms.pdf
- 10.5555/2835577.2835578 · doi:10.5555/2835577.2835578
Cited by
Questioning the AI: Informing Design Practices for Explainable AI User Experiences
Liao, Q. V., Gruen, D., & Miller, S. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376590 · Cited by 925
“[30] proposed a stage-based participatory design process, which guides product-specific needs specification–what to explain, followed by iterative design of solutions–how to explain.”
BackgroundInfluentialDesigning Theory-Driven User-Centric Explainable AI
Wang, D., Yang, Q., Abdul, A., & Lim, B. Y. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300831 · Cited by 906
A Multidisciplinary Survey and Framework for Design and Evaluation of Explainable AI Systems
Mohseni, S., Zarei, N., & Ragan, E. D. (2021) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3387166 · Cited by 785
“Another example is the occasional case of using the terms Transparent System and Explainable System interchangeably in HCI research [56], while others clarify that explainability is not equivalent to transparency because it does not require knowing the flow of the bits in the AI decision-making…”
BackgroundMethodInfluentialA Multidisciplinary Survey and Framework for Design and Evaluation of Explainable AI Systems
Mohseni, S., Zarei, N., & Ragan, E. D. (2018) · arXiv · doi:10.48550/arxiv.1811.11839 · Cited by 783
“of surveyed papers, we propose a framework that provides a step-by-step design and evaluation plan for a multidisciplinary team of designers for building real-world XAI systems. Unlike Eiband et al. [47], we do not make the assumption of adding transparency to an existing intelligent interface and do not limit the eva…”
BackgroundFelzmann, H., Villaronga, E. F., Lutz, C., & Tamò-Larrieux, A. (2019) · Big Data & Society · doi:10.1177/2053951719860542 · Cited by 447
“From an industry point of view, this is the same case (Eiband et al., 2018).”
Background
Metadata
- Authors
- Eiband, M., Schneider, H., Bilandzic, M., Fazekas-Con, J., Haug, M., & Hussmann, H.
- Affiliations
- Ludwig-Maximilians-Universität München (DE)
- Freeletics GmbH, Munich, Germany
- Venue
- 23rd International Conference on Intelligent User Interfaces
- Publisher
- ACM
- Published
- 2018-03-05
- Pages
- 211-223
- OpenAlex
- W2790505917
- Cited by
- 250
- References
- 40
- Citations by year
- 2026: 46 · 2025: 37 · 2024: 27 · 2023: 25 · 2022: 33 · 2021: 31 · 2020: 28 · 2019: 21 · 2018: 2
- FWCI
- 8.34
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- ACM International Conference on Intelligent User Interfaces (IUI) (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Physical Sciences › Computer Science › Human-Computer Interaction › Usability and User Interface Design
- Physical Sciences › Computer Science › Human-Computer Interaction › Innovative Human-Technology Interaction
- Keywords
- intelligent systems, user interface design, participatory design, multiple stakeholders, explainable AI, exercise instructors, recommender systems, design guidelines
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.