Explaining Recommendations in E-Learning: Effects on Adolescents' Trust
Open accessThe DOI matches its Crossref record; 28 of the 28 references it cites were found in a registry or an index.
Abstract
In the scope of explainable artificial intelligence, explanation techniques are heavily studied to increase trust in recommender systems. However, studies on explaining recommendations typically target adults in e-commerce or media contexts; e-learning has received less research attention. To address these limits, we investigated how explanations affect adolescents’ initial trust in an e-learning platform that recommends mathematics exercises with collaborative filtering. In a randomized controlled experiment with 37 adolescents, we compared real explanations with placebo and no explanations. Our results show that real explanations significantly increased initial trust when trust was measured as a multidimensional construct of competence, benevolence, integrity, intention to return, and perceived transparency. Yet, this result did not hold when trust was measured one-dimensionally. Furthermore, not all adolescents attached equal importance to explanations and trust scores were high overall. These findings underline the need to tailor explanations and suggest that dynamically learned factors may be more important than explanations for building initial trust. To conclude, we thus reflect upon the need for explanations and recommendations in e-learning in low-stakes and high-stakes situations.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Barredo Arrieta, A., Díaz-Rodríguez, N., Del Ser, J., Bennetot, A., Tabik, S., Barbado, A., Garcia, S., Gil-Lopez, S., Molina, D., Benjamins, R., et al. (2020) · Information Fusion · doi:10.1016/j.inffus.2019.12.012 · Cited by 10,423
Trust in Automation: Designing for Appropriate Reliance
Lee, J. D., & See, K. A. (2004) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/hfes.46.1.50_30392 · Cited by 6,605 · cited 1×
“On the one hand, it has been defined in many different ways, depending on the field or context [46] and entailing different themes such as competence, benevolence, and reliance [7, 13, 14, 30, 44, 54].”
BackgroundPeeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI)
Adadi, A., & Berrada, M. (2018) · IEEE Access · doi:10.1109/access.2018.2870052 · Cited by 6,294
Lipton, Z. C. (2018) · Queue · doi:10.1145/3236386.3241340 · Cited by 5,427
Explanation in Artificial Intelligence: Insights from the Social Sciences
Miller, T. (2017) · arXiv · doi:10.48550/arxiv.1706.07269 · Cited by 5,338
Explanation in artificial intelligence: Insights from the social sciences
Miller, T. (2019) · Artificial Intelligence · doi:10.1016/j.artint.2018.07.007 · Cited by 5,302
Trust in Automation: Integrating Empirical Evidence on Factors That Influence Trust
Hoff, K. A., & Bashir, M. (2015) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720814547570 · Cited by 2,817
Explaining Explanations: An Overview of Interpretability of Machine Learning
Gilpin, L. H., Bau, D., Yuan, B. Z., Bajwa, A., Specter, M., & Kagal, L. (2018) · 2018 IEEE 5th International Conference on Data Science and Advanced Analytics (DSAA) · doi:10.1109/dsaa.2018.00018 · Cited by 2,507 · cited 1×
“As a consequence, it is hard to find all-embracing definitions for XAI and concepts like ‘explainability’, ‘interpretability’, ‘understandability’ and ‘intelligibility’ [20, 28, 45].”
BackgroundPingouin: statistics in Python
Vallat, R. (2018) · Journal of Open Source Software · doi:10.21105/joss.01026 · Cited by 1,780
Explaining collaborative filtering recommendations
Herlocker, J. L., Konstan, J. A., & Riedl, J. (2000) · Proceedings of the 2000 ACM conference on Computer supported cooperative work · doi:10.1145/358916.358995 · Cited by 1,731 · cited 8×
“[33] compared several explanation designs for collaborative filtering recommenders to increase acceptance of recommendations.”
BackgroundMethodInfluentialDARPA's Explainable Artificial Intelligence Program
Gunning, D., & Aha, D. W. (2019) · AI Magazine · doi:10.1609/aimag.v40i2.2850 · Cited by 1,360 · cited 1×
“Explainable artificial intelligence (XAI) is an umbrella term for techniques that try to explain the logic behind algorithmic decision-making, such that people can understand it, grow appropriate trust in the algorithm, and detect potential biases [32].”
BackgroundA survey of trust in internet applications
Grandison, T., & Sloman, M. (2000) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2000.5340804 · Cited by 1,157 · cited 2×
“This further justifies the presence of competence in many definitions of trust [30, 54, 75].”
BackgroundA Multidisciplinary Survey and Framework for Design and Evaluation of Explainable AI Systems
Mohseni, S., Zarei, N., & Ragan, E. D. (2018) · arXiv · doi:10.48550/arxiv.1811.11839 · Cited by 783
Explainable Recommendation: A Survey and New Perspectives
Zhang, Y., & Chen, X. (2020) · Foundations and Trends® in Information Retrieval · doi:10.1561/1500000066 · Cited by 755
Abdul, A., Vermeulen, J., Wang, D., Lim, B. Y., & Kankanhalli, M. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3174156 · Cited by 743
Langer, E. J., Blank, A., & Chanowitz, B. (1978) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.36.6.635 · Cited by 694
Vellido, A. (2020) · Neural Computing and Applications · doi:10.1007/s00521-019-04051-w · Cited by 635
The effects of transparency on trust in and acceptance of a content-based art recommender
Cramer, H., Evers, V., Ramlal, S., van Someren, M., Rutledge, L., Stash, N., Aroyo, L., & Wielinga, B. (2008) · User Modeling and User-Adapted Interaction · doi:10.1007/s11257-008-9051-3 · Cited by 448 · cited 8×
“[14, 33], and increase people’s trust in the recommender [8, 61].”
BackgroundInfluentialThe Role of Explanations on Trust and Reliance in Clinical Decision Support Systems
Bussone, A., Stumpf, S., & O'Sullivan, D. (2015) · 2015 International Conference on Healthcare Informatics · doi:10.1109/ichi.2015.26 · Cited by 441 · cited 1×
“At the same time, trust research is somewhat controversial [17] because optimizing systems’ designs to grow trust might lead to inappropriate trust, which can entail undesirable effects like misusing technology [11, 50].”
BackgroundToo much, too little, or just right? Ways explanations impact end users' mental models
Kulesza, T., Stumpf, S., Burnett, M., Yang, S., Kwan, I., & Wong, W.-K. (2013) · 2013 IEEE Symposium on Visual Languages and Human Centric Computing · doi:10.1109/vlhcc.2013.6645235 · Cited by 388 · cited 1×
“Furthermore, designing explanations involves making several trade-offs [41].”
BackgroundTrust building with explanation interfaces
Pu, P., & Chen, L. (2006) · Proceedings of the 11th international conference on Intelligent user interfaces · doi:10.1145/1111449.1111475 · Cited by 300 · cited 6×
“Trusting automated systems has been found essential for adopting them [7, 61].”
BackgroundInfluentialHuman-Centered Explainable AI: Towards a Reflective Sociotechnical Approach
Ehsan, U., & Riedl, M. O. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-60117-1_33 · Cited by 274
Bringing Transparency Design into Practice
Eiband, M., Schneider, H., Bilandzic, M., Fazekas-Con, J., Haug, M., & Hussmann, H. (2018) · 23rd International Conference on Intelligent User Interfaces · doi:10.1145/3172944.3172961 · Cited by 250 · cited 1×
“Designing explanations for recommendations brings challenges concerning what and how to explain [23].”
BackgroundLet Me Explain: Impact of Personal and Impersonal Explanations on Trust in Recommender Systems
Kunkel, J., Donkers, T., Michael, L., Barbu, C.-M., & Ziegler, J. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300717 · Cited by 156 · cited 3×
“While previous studies typically focused on recommenders for movies or e-commerce [e.g., 42], research in an e-learning context is limited [5, 15].”
BackgroundThe Impact of Placebic Explanations on Trust in Intelligent Systems
Eiband, M., Buschek, D., Kremer, A., & Hussmann, H. (2019) · Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290607.3312787 · Cited by 152
Millecamp, M., Htun, N. N., Conati, C., & Verbert, K. (2019) · Proceedings of the 24th International Conference on Intelligent User Interfaces · doi:10.1145/3301275.3302313 · Cited by 148
Self-citationVisual, textual or hybrid: the effect of user expertise on different explanations
Szymanski, M., Millecamp, M., & Verbert, K. (2021) · 26th International Conference on Intelligent User Interfaces · doi:10.1145/3397481.3450662 · Cited by 131
Self-citationComplementing educational recommender systems with open learner models
Abdi, S., Khosravi, H., Sadiq, S., & Gasevic, D. (2020) · Proceedings of the Tenth International Conference on Learning Analytics & Knowledge · doi:10.1145/3375462.3375520 · Cited by 68
Cited by
Towards Human-Centered Explainable AI: A Survey of User Studies for Model Explanations
Rong, Y., Leemann, T., Nguyen, T.-T., Fiedler, L., Qian, P., Unhelkar, V., Seidel, T., Kasneci, G., & Kasneci, E. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3331846 · Cited by 278
Human-centered evaluation of explainable AI applications: a systematic review
Kim, J., Maathuis, H., & Sent, D. (2024) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2024.1456486 · Cited by 130
Improving Human Verification of LLM Reasoning through Interactive Explanation Interfaces
Zhou, R., Nguyen, G., Kharya, N., Nguyen, A., & Agarwal, C. (2026) · Proceedings of the 31st International Conference on Intelligent User Interfaces · doi:10.1145/3742413.3789134 · Cited by 2
“…which we use as a proxy for cognitive effort, with longer times potentially indicating greater difficulty in comprehension or in identifying error; iv) Interaction Behavior [42]: All interface interactions, including play, pause, step forward, and step back, are recorded at the click level.”
Metadata
- Authors
- Ooge, J., Kato, S., & Verbert, K.
- Venue
- 27th International Conference on Intelligent User Interfaces
- Publisher
- ACM
- Published
- 2022-03-22
- Pages
- 93-105
- OpenAlex
- W4220699816
- Cited by
- 51
- References
- 28
- FWCI
- 6.59
- Type
- conference paper
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.