The DOI matches its Crossref record; 30 of the 31 references it cites were found in a registry or an index.
Tell me more?: the effects of mental model soundness on personalizing an intelligent agent
Abstract
What does a user need to know to productively work with an intelligent agent? Intelligent agents and recommender systems are gaining widespread use, potentially creating a need for end users to understand how these systems operate in order to fix their agent's personalized behavior. This paper explores the effects of mental model soundness on such personalization by providing structural knowledge of a music recommender system in an empirical study. Our findings show that participants were able to quickly build sound mental models of the recommender system's reasoning, and that participants who most improved their mental models during the study were significantly more likely to make the recommender operate to their satisfaction. These results suggest that by helping end users understand a system's reasoning, intelligent agents may elicit more and better feedback, thus more closely aligning their output with each user's intentions.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“People decide when to rely on an AI by using their mental models [3, 30], or internal representations, of how the AI tends to behave: when it is most accurate, when it is most likely to fail, etc.”
From Improving Human-AI Collaboration With Descriptions of AI Behavior (2023) · cites this paper as [30]
Citations
Cites
Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research
(1988) · Advances in Psychology · doi:10.1016/s0166-4115(08)62386-9 · Cited by 15,003
Self-efficacy: Toward a unifying theory of behavioral change
Bandura, A. (1978) · Advances in Behaviour Research and Therapy · doi:10.1016/0146-6402(78)90002-4 · Cited by 10,586
Johnson-Laird, P. N. (1983) · Cited by 5,190
LaVan, H., & Carley, C. (1984) · Simulation & Games · doi:10.1177/0037550084152005 · Cited by 3,151
Kaplan, B., DeGross, J. I., Wood-Harper, A. T., Wastell, D. G., & Truex, D. P., III (2004) · IFIP International Federation for Information Processing/IFIP · doi:10.1007/b115738 · Cited by 3,139
Explaining collaborative filtering recommendations
Herlocker, J. L., Konstan, J. A., & Riedl, J. (2000) · Proceedings of the 2000 ACM conference on Computer supported cooperative work · doi:10.1145/358916.358995 · Cited by 1,731
Some observations on mental models
Norman, D. A. (1983) · Medical Entomology and Zoology · Cited by 1,549
Application of Social Cognitive Theory to Training for Computer Skills
Compeau, D., & Higgins, C. A. (1995) · Information Systems Research · doi:10.1287/isre.6.2.118 · Cited by 1,503
The General Inquirer: A Computer Approach to Content Analysis
Green, B. F., Stone, P. J., Dunphy, D. C., Smith, M. S., & Ogilvie, D. M. (1967) · American Educational Research Journal · doi:10.2307/1161774 · Cited by 1,013
Hancock, P. A., & Meshkati, N. (1988) · Cited by 954
The General Inquirer: A Computer Approach to Content Analysis.
Hartman, J. J., Stone, P. J., Dunphy, D. C., Smith, M. S., & Ogilvia, D. M. (1967) · American Sociological Review · doi:10.2307/2092070 · Cited by 723
Why and why not explanations improve the intelligibility of context-aware intelligent systems
Lim, B. Y., Dey, A. K., & Avrahami, D. (2009) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1518701.1519023 · Cited by 552 · cited 2×
“descriptions of the agent’s reasoning [13,15], visual depictions of the assistant’s known correct predictions versus its known failures [26], and electronic “door tags” displaying predictions of worker interruptibility with the reasons underlying each prediction (e.”
BackgroundThe role of transparency in recommender systems
Sinha, R. R., & Swearingen, K. (2002) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/506443.506619 · Cited by 513
Interaction Design: Beyond Human-Computer Interaction. Second Edition
Rogers, Y., Sharp, H., & Preece, J. J. (2007) · Wiley eBooks · Cited by 390
Talbot, J., Lee, B., Kapoor, A., & Tan, D. S. (2009) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/1518701.1518895 · Cited by 231
Human model evaluation in interactive supervised learning
Fiebrink, R., Cook, P. R., & Trueman, D. (2011) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1978942.1978965 · Cited by 207
Wilfong, J. D. (2004) · Computers in Human Behavior · doi:10.1016/j.chb.2004.03.020 · Cited by 185
Interacting meaningfully with machine learning systems: Three experiments
Stumpf, S., Rajaram, V., Li, L., Wong, W.-K., Burnett, M., Dietterich, T., Sullivan, E., & Herlocker, J. (2009) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2009.03.004 · Cited by 184
Self-citationEffective explanations of recommendations
Tintarev, N., & Masthoff, J. (2007) · ACM Conference on Recommender Systems (RecSys) · doi:10.1145/1297231.1297259 · Cited by 162
Toolkit to support intelligibility in context-aware applications
Lim, B. Y., & Dey, A. K. (2010) · Proceedings of the 12th ACM international conference on Ubiquitous computing · doi:10.1145/1864349.1864353 · Cited by 143
Interactive optimization for steering machine classification
Kapoor, A., Lee, B., Tan, D. S., & Horvitz, E. (2010) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/1753326.1753529 · Cited by 138
Tullio, J., Dey, A. K., Chalecki, J., & Fogarty, J. A. (2007) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/1240624.1240630 · Cited by 137
Toward harnessing user feedback for machine learning
Stumpf, S., Rajaram, V., Li, L., Burnett, M., Dietterich, T. G., Sullivan, E., Drummond, R., & Herlocker, J. L. (2007) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/1216295.1216316 · Cited by 134
Self-citationExplanatory Debugging: Supporting End-User Debugging of Machine-Learned Programs
Kulesza, T., Stumpf, S., Burnett, M., Wong, W.-K., Riche, Y., Moore, T., Oberst, I., Shinsel, A., & McIntosh, K. (2010) · IEEE Symposium on Visual Languages / Human-Centric Computing Languages and Environments · doi:10.1109/vlhcc.2010.15 · Cited by 124
Self-citationFixing the program my computer learned: barriers for end users, challenges for the machine
Kulesza, T., Wong, W.-K., Stumpf, S., Perona, S., White, R., Burnett, M. M., Oberst, I., & Ko, A. J. (2009) · Proceedings of the 14th international conference on Intelligent user interfaces · doi:10.1145/1502650.1502678 · Cited by 83
Self-citationSmalltalk scaffolding: a case study of minimalist instruction
Rosson, M. B., Carrol, J. M., & Bellamy, R. (1990) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/97243.97319 · Cited by 80
Confidence Displays and Training in Recommender Systems.
McNee, S. M., Lam, S. K., Guetzlaff, C., Konstan, J. A., & Riedl, J. (2003) · International Conference on Human-Computer Interaction · Cited by 56
Examining multiple potential models in end-user interactive concept learning
Amershi, S., Fogarty, J. A., Kapoor, A., & Tan, D. S. (2010) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/1753326.1753531 · Cited by 40
The relationship of instrumental and expressive traits with computer anxiety
Bozionelos, N. (2001) · Personality and Individual Differences · doi:10.1016/s0191-8869(00)00198-7 · Cited by 23
Flagged first, then by citations
1 of the 31 references couldn’t be matched to a known paper.
Research works printed but not linked: 1
- Benedek , J. and Miner , T . Measuring desirability: New methods for evaluating desirability in a usability lab setting . In Proc. Usability Professionals Association International Conference ( 2002 ). Benedek, J. and Miner, T. Measuring desirability: New methods for evaluating desirability in a usability lab setting. In Proc. Usability Professionals Association International Conference (2002).
Cited by
Sorted by citations
Power to the People: The Role of Humans in Interactive Machine Learning
Amershi, S., Cakmak, M., Knox, W. B., & Kulesza, T. (2014) · AI Magazine · doi:10.1609/aimag.v35i4.2513 · Cited by 1,084
Self-citationAbdul, A., Vermeulen, J., Wang, D., Lim, B. Y., & Kankanhalli, M. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3174156 · Cited by 743
Working with Machines: The Impact of Algorithmic and Data-Driven Management on Human Workers
Lee, M. K., Kusbit, D., Metsky, E., & Dabbish, L. (2015) · Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems · doi:10.1145/2702123.2702548 · Cited by 823
A Multidisciplinary Survey and Framework for Design and Evaluation of Explainable AI Systems
Mohseni, S., Zarei, N., & Ragan, E. D. (2021) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3387166 · Cited by 785
Principles of Explanatory Debugging to Personalize Interactive Machine Learning
Kulesza, T., Burnett, M., Wong, W.-K., & Stumpf, S. (2015) · Proceedings of the 20th International Conference on Intelligent User Interfaces · doi:10.1145/2678025.2701399 · Cited by 576
Self-citation
Metadata
- Authors
- Kulesza, T., Stumpf, S., Burnett, M., & Kwan, I.
- Affiliations
- Oregon State University (US)
- City, University of London (GB)
- Venue
- Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
- Publisher
- ACM
- Published
- 2012-05-05
- Pages
- 1-10
- OpenAlex
- W2168747479
- Cited by
- 225
- References
- 31
- Citations by year
- 2026: 14 · 2025: 25 · 2024: 22 · 2023: 25 · 2022: 24 · 2021: 24 · 2020: 35 · 2019: 16 · 2018: 14 · 2017: 6 · 2016: 9 · 2015: 5 · 2014: 3 · 2013: 1 · 2012: 1
- FWCI
- 2.52
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- CHI Conference on Human Factors in Computing Systems (publishedVersion)
- City Research Online (City University London) (acceptedVersion)
- ENLIGHTEN (Jurnal Bimbingan dan Konseling Islam) (submittedVersion)
- Funders
- National Science Foundation (0803487)
- Topics
- Physical Sciences › Computer Science › Information Systems › Recommender Systems and Techniques
- Physical Sciences › Computer Science › Human-Computer Interaction › Usability and User Interface Design
- Physical Sciences › Computer Science › Information Systems › Information Retrieval and Search Behavior
- Keywords
- mental models, intelligent agents, recommender systems, music recommendation, personalization, user satisfaction, system transparency
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
