Assessing demand for intelligibility in context-aware applications
The DOI matches its Crossref record; 28 of the 28 references it cites were found in a registry or an index.
Abstract
Intelligibility can help expose the inner workings and inputs of context-aware applications that tend to be opaque to users due to their implicit sensing and actions. However, users may not be interested in all the information that the applications can produce. Using scenarios of four real-world applications that span the design space of context-aware computing, we conducted two experiments to discover what information users are interested in. In the first experiment, we elicit types of information demands that users have and under what moderating circumstances they have them. In the second experiment, we verify the findings by soliciting users about which types they would want to know and establish whether receiving such information would satisfy them. We discuss why users demand certain types of information, and provide design implications on how to provide different intelligibility types to make context-aware applications intelligible and acceptable to users.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Consistent with our initial expectations and the prior literature on transparency [28,49,85], users anticipated a preference for transparency before using the system.”
From Progressive Disclosure: When, Why, and How Do Users Want Algorithmic Transparency Information? (2020) · cites this paper as [49]
In the citing paper
“We first confirm other work showing that as anticipated, transparency is beneficial overall [22,49,50].”
From Progressive Disclosure: When, Why, and How Do Users Want Algorithmic Transparency Information? (2020) · cites this paper as [49]
In the citing paper
“Consistent with our initial expectations and the prior literature on transparency [26,48], users anticipated a preference for transparency before using the system.”
From Progressive disclosure (2019) · cites this paper as [48]
In the citing paper
“The questions in this section were adapted from work by Lim [43].”
From It’s Just Not That Simple: An Empirical Study of the Accuracy-Explainability Trade-off in Machine Learning for Public Policy (2022) · cites this paper as [43]
Citations
Cites
Basics of Qualitative Research: Grounded Theory Procedures and Techniques.
Ellis, C., Strauss, A., & Corbin, J. (1992) · Contemporary Sociology · doi:10.2307/2074814 · Cited by 29,249
(1990) · Cited by 5,210
Orlikowski, W. J. (2000) · Organization Science · doi:10.1287/orsc.11.4.404.14600 · Cited by 4,760
Dey, A. K., Abowd, G. D., & Salber, D. (2001) · Human–Computer Interaction · doi:10.1207/s15327051hci16234_02 · Cited by 2,951 · cited 1×
“Context-aware applications use context – information about the state of people, places, and objects relevant to users and their activities [9] — to make inferences as to how to adapt their behavior.”
BackgroundSelf-citationExplaining collaborative filtering recommendations
Herlocker, J. L., Konstan, J. A., & Riedl, J. (2000) · Proceedings of the 2000 ACM conference on Computer supported cooperative work · doi:10.1145/358916.358995 · Cited by 1,731 · cited 1×
“This approach has been employed in several domains including decision making [11], recommender systems [17], end-user debugging [18, 19], and systems that model user profiles [7], with the goal of increasing user trust and acceptance of these systems.”
MethodCyberguide: A mobile context‐aware tour guide
Abowd, G. D., Atkeson, C. G., Hong, J., Long, S., Kooper, R., & Pinkerton, M. (1997) · Wireless Networks · doi:10.1023/a:1019194325861 · Cited by 1,365 · cited 1×
“Tour Guide: CyberGuide CyberGuide [1] is a mobile context-aware tour guide that uses context to recommend attractions to a user.”
BackgroundThe role of trust in automation reliance
Dzindolet, M. T., Peterson, S. A., Pomranky, R. A., Pierce, L. G., & Beck, H. P. (2003) · International Journal of Human-Computer Studies · doi:10.1016/s1071-5819(03)00038-7 · Cited by 1,276 · cited 1×
“This approach has been employed in several domains including decision making [11], recommender systems [17], end-user debugging [18, 19], and systems that model user profiles [7], with the goal of increasing user trust and acceptance of these systems.”
MethodMUIR, B. M. (1994) · Ergonomics · doi:10.1080/00140139408964957 · Cited by 873 · cited 1×
“Such a lack of application intelligibility can lead users to mistrust the system, misuse it, or abandon it altogether [ 22 ].”
BackgroundDigital family portraits: supporting peace of mind for extended family members
Mynatt, E. D., Rowan, J., Craighill, S., & Jacobs, A. (2001) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/365024.365126 · Cited by 582 · cited 1×
“Remote Monitoring: Digital Family Portrait The Digital Family Portrait [23] leverages a picture frame to present the current status of an elderly family member as he or she goes through daily life living independently in her home, to remote loved ones.”
BackgroundFroehlich (2009) · doi:10.1145/1518701.1518861 · Cited by 563
Why and why not explanations improve the intelligibility of context-aware intelligent systems
Lim, B. Y., Dey, A. K., & Avrahami, D. (2009) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1518701.1519023 · Cited by 552 · cited 6×
“Using the open coding method of grounded theory [26], and drawing from question types employed previously [3, 20], we derived a set of information types that users are interested in for context-aware applications.”
BackgroundMethodResultInfluentialSelf-citationExplanations From Intelligent Systems: Theoretical Foundations and Implications for Practice1
Gregor, S., & Benbasat, I. (1999) · MIS Quarterly · doi:10.2307/249487 · Cited by 535 · cited 1×
“Gregor and Benbasat’s [16] meta-review investigates explanation types that users of knowledge-based systems (KBS) would like to have.”
BackgroundIntelligibility and Accountability: Human Considerations in Context-Aware Systems
Bellotti, V., & Edwards, K. (2001) · Human–Computer Interaction · doi:10.1207/s15327051hci16234_05 · Cited by 471 · cited 3×
“, [5, 6, 10]), our participants were interested in knowing how to control applications.”
BackgroundThe human experience [of ubiquitous computing]
Abowd, G. D., Mynatt, E. D., & Rodden, T. (2002) · IEEE Pervasive Computing · doi:10.1109/mprv.2002.993144 · Cited by 352 · cited 1×
“While adaptive agents and KBS are similar to context-aware applications (which may also use agents or knowledge bases and rules), they are workoriented, while context-aware applications are targeted for everyday use, for many more situations and a wider range of users, and under more situations [ 2 ].”
BackgroundPredicting human interruptibility with sensors
Fogarty, J., Hudson, S. E., Atkeson, C. G., Avrahami, D., Forlizzi, J., Kiesler, S., Lee, J. C., & Yang, J. (2005) · ACM Transactions on Computer-Human Interaction · doi:10.1145/1057237.1057243 · Cited by 344 · cited 1×
“Our application uses the responsiveness prediction to determine the subject’s interruptibility [12], and either forwards or suppresses incoming IM messages.”
MethodIs Context-Aware Computing Taking Control away from the User? Three Levels of Interactivity Examined
Barkhuus, L., & Dey, A. (2003) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-39653-6_12 · Cited by 303 · cited 3×
“, [5, 6, 10]), our participants were interested in knowing how to control applications.”
BackgroundSelf-citationCybreMinder: A Context-Aware System for Supporting Reminders
Dey, A. K., & Abowd, G. D. (2000) · doi:10.5555/647986.757284 · Cited by 291
Self-citationSelection Bias in Web Surveys and the Use of Propensity Scores
Schonlau, M., van Soest, A., Kapteyn, A., & Couper, M. (2009) · Sociological Methods & Research · doi:10.1177/0049124108327128 · Cited by 268 · cited 1×
“Third, since we drew our participants from the Internet, and a relatively new platform, Mechanical Turk, our population is not a representative sample of the general public (see [25]).”
BackgroundiCAP: Interactive Prototyping of Context-Aware Applications
Dey, A. K., Sohn, T., Streng, S., & Kodama, J. (2006) · Lecture Notes in Computer Science · doi:10.1007/11748625_16 · Cited by 237
Self-citationToward establishing trust in adaptive agents
Glass, A., McGuinness, D. L., & Wolverton, M. (2008) · Proceedings of the 13th international conference on Intelligent user interfaces · doi:10.1145/1378773.1378804 · Cited by 211 · cited 2×
“McGuinness and colleagues [14, 21] have identified information need factors that influence the level of trust in adaptive agents.”
BackgroundStrauss, A., & Corbin, J. M. (1990) · Journal of International Crisis and Risk Communication Research · Cited by 155
Graesser, A. C., & McMahen, C. L. (1993) · Journal of Educational Psychology · doi:10.1037//0022-0663.85.1.136 · Cited by 132 · cited 1×
“Since people ask information seeking questions due to cognitive disequilibrium [15] and to correct knowledge deficits [27], we believe that satisfying these information demands through intelligibility can lead to better satisfaction when using these applications and improved adoption and acceptance.”
BackgroundFinding causes of program output with the Java Whyline
Ko, A. J., & Myers, B. A. (2009) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1518701.1518942 · Cited by 123 · cited 1×
“This approach has been employed in several domains including decision making [11], recommender systems [17], end-user debugging [18, 19], and systems that model user profiles [7], with the goal of increasing user trust and acceptance of these systems.”
MethodTowards improving trust in context-aware systems by displaying system confidence
Antifakos, S., Kern, N., Schiele, B., & Schwaninger, A. (2005) · Proceedings of the 7th international conference on Human computer interaction with mobile devices & services · doi:10.1145/1085777.1085780 · Cited by 95 · cited 1×
“Using the open coding method of grounded theory [26], and drawing from question types employed previously [3, 20], we derived a set of information types that users are interested in for context-aware applications.”
MethodResponsiveness in instant messaging: predictive models supporting inter-personal communication
Avrahami, D., & Hudson, S. E. (2006) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1124772.1124881 · Cited by 84 · cited 1×
“We designed the instant messenger (IM) auto-notification plugin based on recent [ 4 ] work on a predictive model to determine how long a buddy would take to respond to a message.”
MethodFixing the program my computer learned: barriers for end users, challenges for the machine
Kulesza, T., Wong, W.-K., Stumpf, S., Perona, S., White, R., Burnett, M. M., Oberst, I., & Ko, A. J. (2009) · Proceedings of the 14th international conference on Intelligent user interfaces · doi:10.1145/1502650.1502678 · Cited by 83 · cited 2×
“This approach has been employed in several domains including decision making [11], recommender systems [17], end-user debugging [18, 19], and systems that model user profiles [7], with the goal of increasing user trust and acceptance of these systems.”
MethodCheverst, K., Byun, H. E., Fitton, D., Sas, C., Kray, C., & Villar, N. (2005) · User Modeling and User-Adapted Interaction · doi:10.1007/s11257-005-1269-8 · Cited by 81
A Categorization of Explanation Questions for Task Processing Systems.
McGuinness, D. L., Glass, A., Wolverton, M. J., & da Silva, P. P. (2007) · Cited by 12
Research works printed but not linked: 2
- 10.1037/0022-0663.85.1.136 · doi:10.1037/0022-0663.85.1.136 · Paper page
- Van der Meik , H. ( 1987 ). Assumptions of information-seeking questions . Questioning Exchange 1 , 111 -- 118 . Van der Meik, H. (1987). Assumptions of information-seeking questions. Questioning Exchange 1, 111--118.
Cited by
Explanation in artificial intelligence: Insights from the social sciences
Miller, T. (2019) · Artificial Intelligence · doi:10.1016/j.artint.2018.07.007 · Cited by 5,302
Guidelines for Human-AI Interaction
Amershi, S., Weld, D., Vorvoreanu, M., Fourney, A., Nushi, B., Collisson, P., Suh, J., Iqbal, S., Bennett, P. N., Inkpen, K., et al. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300233 · Cited by 2,439
“Similarly, researchers have for decades studied human interaction with intelligent context-aware computing systems including how to design for understandability and control of the underlying sensing systems [3, 23] and how to support ambiguity resolution [10].”
BackgroundAli, S., Abuhmed, T., El-Sappagh, S., Muhammad, K., Alonso-Moral, J. M., Confalonieri, R., Guidotti, R., Del Ser, J., Díaz-Rodríguez, N., & Herrera, F. (2023) · Information Fusion · doi:10.1016/j.inffus.2023.101805 · Cited by 1,762
Questioning the AI: Informing Design Practices for Explainable AI User Experiences
Liao, Q. V., Gruen, D., & Miller, S. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376590 · Cited by 925
“We added three more types based on [44, 55, 56]– questions regarding model Input (training data), Output, and Performance.”
BackgroundMethodInfluentialDesigning Theory-Driven User-Centric Explainable AI
Wang, D., Yang, Q., Abdul, A., & Lim, B. Y. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300831 · Cited by 906
“decision aids [93], tutoring systems [32], intelligible context-aware systems [66], etc.”
BackgroundSelf-citation
Metadata
- Authors
- Lim, B. Y., & Dey, A. K.
- Affiliations
- Carnegie Mellon University (US)
- Venue
- Proceedings of the 11th international conference on Ubiquitous computing
- Publisher
- ACM
- Published
- 2009-09-30
- Pages
- 195-204
- OpenAlex
- W2131251708
- Cited by
- 246
- References
- 28
- Citations by year
- 2026: 23 · 2025: 18 · 2024: 15 · 2023: 27 · 2022: 25 · 2021: 30 · 2020: 16 · 2019: 18 · 2018: 18 · 2017: 8 · 2016: 5 · 2015: 8 · 2014: 3 · 2013: 12 · 2012: 7
- FWCI
- 5.08
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp) (publishedVersion)
- Funders
- National Science Foundation
- Agency for Science, Technology and Research
- Topics
- Physical Sciences › Computer Science › Human-Computer Interaction › Usability and User Interface Design
- Physical Sciences › Computer Science › Human-Computer Interaction › Innovative Human-Technology Interaction
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Keywords
- context-aware systems, intelligibility, transparency, user acceptance, user-centered design
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.