Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems
Open accessAbstract
Course selection is a crucial activity for students as it directly impacts their workload and performance. It is also time-consuming, prone to subjectivity, and often carried out based on incomplete information. This task can, nevertheless, be assisted with computational tools, for instance, by predicting performance based on historical data. We investigate the effects of showing grade predictions to students through an interactive visualization tool. A qualitative study suggests that in the presence of predictions, students may focus too much on maximizing their performance, to the detriment of other factors such as the workload. A follow-up quantitative study explored whether these effects are mitigated by changing how predictions are conveyed. Our observations suggest the presence of a framing effect that induces students to put more effort into course selection when faced with more specific predictions. We discuss these and other findings and outline considerations for designing better data-driven course selection tools.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The MATCH model focuses on individual users with an AI system and does not explicitly consider the broader sociotechnical ecosystem.
“An important limitation of the current MATCH model and relevant design is that it focuses on the interaction of individual users with an AI system, while not explicitly considering the broader sociotechnical ecosystem surrounding the user and AI.”
From "It would work for me too": How Online Communities Shape Software Developers' Trust in AI-Powered Code Generation Tools (2024) · cites this paper as [44]
A misalignment between users' trust and AI ability could lead to overreliance and misuse of AI systems, resulting in undesirable consequences.
“A misalignment between the trust of users and the ability of AI could lead to user overreliance and misuse of AI systems, resulting in undesirable consequences [ 21 , 43 , 51 ].”
From "It would work for me too": How Online Communities Shape Software Developers' Trust in AI-Powered Code Generation Tools (2024) · cites this paper as [21]
Cites
Ryan, R. M., & Deci, E. L. (2000) · American Psychologist · doi:10.1037//0003-066x.55.1.68 · Cited by 27,815 · see citing works
A Unified Approach to Interpreting Model Predictions
Lundberg, S., & Lee, S.-I. (2017) · arXiv · doi:10.48550/arxiv.1705.07874 · Cited by 23,707 · see citing works
All Frames Are Not Created Equal: A Typology and Critical Analysis of Framing Effects
Levin, I. P., Schneider, S. L., & Gaeth, G. J. (1998) · Organizational Behavior and Human Decision Processes · doi:10.1006/obhd.1998.2804 · Cited by 2,592 · see citing works
Cleveland, W. S., & McGill, R. (1984) · Journal of the American Statistical Association · doi:10.2307/2288400 · Cited by 1,796 · see citing works
Evaluating amazon’s mechanical turk as a tool for experimental behavioral research
Crump, M. J. C., McDonnell, J. V., & Gureckis, T. M. (2013) · PLoS ONE · doi:10.1371/journal.pone.0057410 · Cited by 1,617 · see citing works
Automation bias: a systematic review of frequency, effect mediators, and mitigators
Goddard, K., Roudsari, A., & Wyatt, J. C. (2012) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2011-000089 · Cited by 1,162 · cited 1× · see citing works
“The lack of interest in the explanations provided by iCoRA might suggest some sort of automation complacency [29] in the students regarding iCoRA’s performance predictions.”
A Theory of Framing and Opinion Formation in Competitive Elite Environments
Chong, D., & Druckman, J. N. (2007) · Journal of Communication · doi:10.1111/j.1460-2466.2006.00331.x · Cited by 1,159 · see citing works
When rewards compete with nature
Ryan, R. M., & Deci, E. L. (2000) · Intrinsic and Extrinsic Motivation · doi:10.1016/b978-012619070-0/50024-6 · Cited by 435 · see citing works
Benefitting InfoVis with Visual Difficulties
Hullman, J., Adar, E., & Shah, P. (2011) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2011.175 · Cited by 159 · see citing works
Boy, J., Pandey, A. V., Emerson, J., Satterthwaite, M., Nov, O., & Bertini, E. (2017) · Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3025453.3025512 · Cited by 104 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Conceptualizing AI literacy: An exploratory review
Ng, D. T. K., Leung, J. K. L., Chu, S. K. W., & Qiao, M. S. (2021) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2021.100041 · Cited by 2,205 · see citing works
Explainable Artificial Intelligence in education
Khosravi, H., Shum, S. B., Chen, G., Conati, C., Tsai, Y.-S., Kay, J., Knight, S., Martinez-Maldonado, R., Sadiq, S., & Gašević, D. (2022) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2022.100074 · Cited by 863 · see citing works
A survey on active learning and human-in-the-loop deep learning for medical image analysis
Budd, S., Robinson, E. C., & Kainz, B. (2021) · Medical Image Analysis · doi:10.1016/j.media.2021.102062 · Cited by 606 · see citing works
Natural language processing of social media as screening for suicide risk
Coppersmith, G., Leary, R., Crutchley, P., & Fine, A. (2018) · Biomedical Informatics Insights · doi:10.1177/1178222618792860 · Cited by 506 · see citing works
Ng, D. T. K., Wu, W., Leung, J. K. L., Chiu, T. K. F., & Chu, S. K. W. (2024) · British Journal of Educational Technology · doi:10.1111/bjet.13411 · Cited by 390 · see citing works
How does generative artificial intelligence impact student creativity?
Habib, S., Vogel, T., Anli, X., & Thorne, E. (2024) · Journal of Creativity · doi:10.1016/j.yjoc.2023.100072 · Cited by 364 · see citing works
Menon, D., & Shilpa, K. (2023) · Heliyon · doi:10.1016/j.heliyon.2023.e20962 · Cited by 359 · see citing works
Kong, S.-C., Man-Yin Cheung, W., & Zhang, G. (2021) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2021.100026 · Cited by 332 · see citing works
Meyer, J., Jansen, T., Schiller, R., Liebenow, L. W., Steinbach, M., Horbach, A., & Fleckenstein, J. (2024) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2023.100199 · Cited by 311 · see citing works
Hornberger, M., Bewersdorff, A., & Nerdel, C. (2023) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2023.100165 · Cited by 276 · see citing works
How transparency modulates trust in artificial intelligence
Zerilli, J., Bhatt, U., & Weller, A. (2022) · Patterns · doi:10.1016/j.patter.2022.100455 · Cited by 257 · see citing works
A survey of visual analytics for Explainable Artificial Intelligence methods
Alicioglu, G., & Sun, B. (2022) · Computers & Graphics · doi:10.1016/j.cag.2021.09.002 · Cited by 221 · see citing works
Examining artificial intelligence literacy among pre-service teachers for future classrooms
Ayanwale, M. A., Adelana, O. P., Molefi, R. R., Adeeko, O., & Ishola, A. M. (2024) · Computers and Education Open · doi:10.1016/j.caeo.2024.100179 · Cited by 238 · see citing works
Ding, A.-C. E., Shi, L., Yang, H., & Choi, I. (2024) · Computers and Education Open · doi:10.1016/j.caeo.2024.100178 · Cited by 228 · see citing works
A systematic review of teaching and learning machine learning in K-12 education
Sanusi, I. T., Oyelere, S. S., Vartiainen, H., Suhonen, J., & Tukiainen, M. (2023) · Education and Information Technologies · doi:10.1007/s10639-022-11416-7 · Cited by 212 · see citing works
Combining human and artificial intelligence for enhanced AI literacy in higher education
Tzirides, A. O. (., Zapata, G., Kastania, N. P., Saini, A. K., Castro, V., Ismael, S. A., You, Y.-L., Santos, T. A. D., Searsmith, D., O'Brien, C., et al. (2024) · Computers and Education Open · doi:10.1016/j.caeo.2024.100184 · Cited by 268 · see citing works
Kong, S.-C., Cheung, M.-Y. W., & Tsang, O. (2024) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2024.100214 · Cited by 167 · see citing works
Wang, T., & Cheng, E. C. K. (2021) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2021.100031 · Cited by 141 · see citing works
Kong, S.-C., Cheung, W. M.-Y., & Zhang, G. (2022) · Computers in Human Behavior Reports · doi:10.1016/j.chbr.2022.100223 · Cited by 135 · see citing works
The explainability paradox: Challenges for xAI in digital pathology
Evans, T., Retzlaff, C. O., Geißler, C., Kargl, M., Plass, M., Müller, H., Kiehl, T.-R., Zerbe, N., & Holzinger, A. (2022) · Future Generation Computer Systems · doi:10.1016/j.future.2022.03.009 · Cited by 135 · see citing works
How Cognitive Biases Affect XAI-assisted Decision-making: A Systematic Review
Bertrand, A., Belloum, R., Eagan, J. R., & Maxwell, W. (2022) · Proceedings of the 2022 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3514094.3534164 · Cited by 129 · see citing works
Integrating generative AI in knowledge building
Chen, B., Zhu, X., & Díaz del Castillo H., F. (2023) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2023.100184 · Cited by 159 · see citing works
Co-design of human-centered, explainable AI for clinical decision support
Panigutti, C., Beretta, A., Fadda, D., Giannotti, F., Pedreschi, D., Perotti, A., & Rinzivillo, S. (2023) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3587271 · Cited by 127 · see citing works
Dikmen, M., & Burns, C. (2022) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2022.102792 · Cited by 116 · see citing works
González-Gonzalo, C., Thee, E. F., Klaver, C. C. W., Lee, A. Y., Schlingemann, R. O., Tufail, A., Verbraak, F., & Sánchez, C. I. (2022) · Progress in Retinal and Eye Research · doi:10.1016/j.preteyeres.2021.101034 · Cited by 103 · see citing works
The Disagreement Deconvolution: Bringing Machine Learning Performance Metrics In Line With Reality
Gordon, M. L., Zhou, K., Patel, K., Hashimoto, T., & Bernstein, M. S. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445423 · Cited by 101 · see citing works
Human-centered evaluation of explainable AI applications: a systematic review
Kim, J., Maathuis, H., & Sent, D. (2024) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2024.1456486 · Cited by 130 · see citing works
Bauducco, S., Pillion, M., Bartel, K., Reynolds, C., Kahn, M., & Gradisar, M. (2024) · Sleep Medicine Reviews · doi:10.1016/j.smrv.2024.101933 · Cited by 102 · see citing works
DeepThInk: Designing and probing human-AI co-creation in digital art therapy
Du, X., An, P., Leung, J., Li, A., Chapman, L. E., & Zhao, J. (2024) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2023.103139 · Cited by 95 · see citing works
The Usability Construct: A Dead End?
Tractinsky, N. (2018) · Human–Computer Interaction · doi:10.1080/07370024.2017.1298038 · Cited by 87 · see citing works
VoxLens: Making Online Data Visualizations Accessible with an Interactive JavaScript Plug-In
Sharif, A., Wang, O. H., Muongchan, A. T., Reinecke, K., & Wobbrock, J. O. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3517431 · Cited by 86 · see citing works
A Survey of Data Quality Requirements That Matter in ML Development Pipelines
Priestley, M., O’donnell, F., & Simperl, E. (2023) · Journal of Data and Information Quality · doi:10.1145/3592616 · Cited by 86 · see citing works
A descriptive study of mental models
PAYNE, S. J. (1991) · Behaviour & Information Technology · doi:10.1080/01449299108924268 · Cited by 81 · see citing works
Cummings, R. E., Monroe, S. M., & Watkins, M. (2024) · Computers and Composition · doi:10.1016/j.compcom.2024.102827 · Cited by 83 · see citing works
Artificial Intelligence in Cardiovascular Care—Part 2: Applications
Jain, S. S., Elias, P., Poterucha, T., Randazzo, M., Lopez Jimenez, F., Khera, R., Perez, M., Ouyang, D., Pirruccello, J., Salerno, M., et al. (2024) · Journal of the American College of Cardiology · doi:10.1016/j.jacc.2024.03.401 · Cited by 74 · see citing works
Explainability pitfalls: beyond dark patterns in explainable AI
Ehsan, U., & Riedl, M. O. (2024) · Patterns · doi:10.1016/j.patter.2024.100971 · Cited by 72 · see citing works
Yanamandra, K., Chen, G. L., Xu, X., Mac, G., & Gupta, N. (2020) · Composites Science and Technology · doi:10.1016/j.compscitech.2020.108318 · Cited by 72 · see citing works
Social gaming: A systematic review
Gonçalves, D., Pais, P., Gerling, K., Guerreiro, T., & Rodrigues, A. (2023) · Computers in Human Behavior · doi:10.1016/j.chb.2023.107851 · Cited by 72 · see citing works
Beyond self-diagnosis: how a chatbot-based symptom checker should respond
You, Y., Tsai, C.-H., Li, Y., Ma, F., Heron, C., & Gui, X. (2023) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3589959 · Cited by 60 · see citing works
Toward AI grading of student problem solutions in introductory physics: A feasibility study
Kortemeyer, G. (2023) · Physical Review Physics Education Research · doi:10.1103/physrevphyseducres.19.020163 · Cited by 59 · see citing works
Seeing Like an Infrastructure: Low-resolution Citizens and the Aadhaar Identification Project
Singh, R., & Jackson, S. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3476056 · Cited by 60 · see citing works
Exploring the effects of human-centered AI explanations on trust and reliance
Scharowski, N., Perrig, S. A. C., Svab, M., Opwis, K., & Brühlmann, F. (2023) · Frontiers in Computer Science · doi:10.3389/fcomp.2023.1151150 · Cited by 57 · see citing works
The need to move away from agential-AI: Empirical investigations, useful concepts and open issues
Cabitza, F., Campagner, A., & Simone, C. (2021) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2021.102696 · Cited by 55 · see citing works
Kim, M., Kim, S., Kim, J., Song, T.-J., & Kim, Y. (2024) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2023.103160 · Cited by 53 · see citing works
Algorithmic harms and digital ageism in the use of surveillance technologies in nursing homes
Berridge, C., & Grigorovich, A. (2022) · Frontiers in Sociology · doi:10.3389/fsoc.2022.957246 · Cited by 47 · see citing works
Integrity-based Explanations for Fostering Appropriate Trust in AI Agents
Mehrotra, S., Jorge, C. C., Jonker, C. M., & Tielman, M. L. (2024) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3610578 · Cited by 51 · see citing works
Psychological Traits and Appropriate Reliance: Factors Shaping Trust in AI
Küper, A., & Krämer, N. (2024) · International Journal of Human–Computer Interaction · doi:10.1080/10447318.2024.2348216 · Cited by 49 · see citing works
Kwon, I., Wang, S.-G., Shin, S.-C., Cheon, Y.-I., Lee, B.-J., Lee, J.-C., Lim, D.-W., Jo, C., Cho, Y., & Shin, B.-J. (2025) · Journal of Voice · doi:10.1016/j.jvoice.2022.07.007 · Cited by 43 · see citing works
Hate Raids on Twitch: Echoes of the Past, New Modalities, and Implications for Platform Governance
Han, C., Seering, J., Kumar, D., Hancock, J. T., & Durumeric, Z. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3579609 · Cited by 46 · see citing works
A Missing Piece in the Puzzle: Considering the Role of Task Complexity in Human-AI Decision Making
Salimzadeh, S., He, G., & Gadiraju, U. (2023) · Proceedings of the 31st ACM Conference on User Modeling, Adaptation and Personalization · doi:10.1145/3565472.3592959 · Cited by 42 · see citing works
Dangol, A., Newman, M., Wolfe, R., Lee, J. H., Kientz, J. A., Yip, J., & Pitt, C. (2024) · Designing Interactive Systems Conference · doi:10.1145/3643834.3661515 · Cited by 42 · see citing works
Cheng, R., Wang, R., Zimmermann, T., & Ford, D. (2024) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3651990 · Cited by 47 · see citing works
Chung, J., Bleich, M., Wheeler, D. C., Winship, J. M., McDowell, B., Baker, D., & Parsons, P. (2021) · Gerontology and Geriatric Medicine · doi:10.1177/23337214211005869 · Cited by 35 · see citing works
CodeWalk: Facilitating Shared Awareness in Mixed-Ability Collaborative Software Development
Potluri, V., Pandey, M., Begel, A., Barnett, M., & Reitherman, S. (2022) · Proceedings of the 24th International ACM SIGACCESS Conference on Computers and Accessibility · doi:10.1145/3517428.3544812 · Cited by 39 · see citing works
Dang, H., Mecke, L., & Buschek, D. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3502141 · Cited by 37 · see citing works
Liew, T. W., Tan, S.-M., Yoo, N. E., Gan, C. L., & Lee, Y. Y. (2023) · Computers in Human Behavior Reports · doi:10.1016/j.chbr.2023.100323 · Cited by 33 · see citing works
Kornfield, R., Stamatis, C. A., Bhattacharjee, A., Pang, B., Nguyen, T., Williams, J. J., Kumar, H., Popowski, S., Beltzer, M., Karr, C. J., et al. (2023) · Internet Interventions · doi:10.1016/j.invent.2023.100667 · Cited by 33 · see citing works
Generation Probabilities Are Not Enough: Uncertainty Highlighting in AI Code Completions
Vasconcelos, H., Bansal, G., Fourney, A., Liao, Q. V., & Vaughan, J. W. (2023) · arXiv · doi:10.48550/arxiv.2302.07248 · Cited by 23
Azimuth: Designing Accessible Dashboards for Screen Reader Users
Srinivasan, A., Harshbarger, T., Hilliker, D., & Mankoff, J. (2023) · The 25th International ACM SIGACCESS Conference on Computers and Accessibility · doi:10.1145/3597638.3608405 · Cited by 17 · see citing works
How chatbots perceive sexting by adolescents
Ricon, T. (2024) · Computers in Human Behavior: Artificial Humans · doi:10.1016/j.chbah.2024.100068 · Cited by 15 · see citing works
Eisbach, S., Mai, O., & Hertel, G. (2024) · New Ideas in Psychology · doi:10.1016/j.newideapsych.2024.101077 · Cited by 12 · see citing works
Representational transformations: Using maps to write essays
Binks, A., Toniolo, A., & Nacenta, M. A. (2022) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2022.102851 · Cited by 12 · see citing works
Resnick, M. S. (2023) · Thinking Skills and Creativity · doi:10.1016/j.tsc.2023.101412 · Cited by 17 · see citing works
Text-to-Image AI as a Catalyst for Semantic Convergence in Creative Collaborations
Lin, P.-Y., Andersen, K., Schmidt, R., Schoenmakers, S., Hofmeyer, H., Pauwels, P., & Ijsselsteijn, W. (2024) · Designing Interactive Systems Conference · doi:10.1145/3643834.3661543 · Cited by 9 · see citing works
A digital therapeutic alliance in digital mental health
Kaveladze, B., & Schueller, S. M. (2023) · Digital Therapeutics for Mental Health and Addiction · doi:10.1016/b978-0-323-90045-4.00009-5 · Cited by 8 · see citing works
Rothschild, A., Booker, J., Davoll, C., Hill, J., Ivey, V., DiSalvo, C., Rydal Shapiro, B., & DiSalvo, B. (2022) · CHI Conference on Human Factors in Computing Systems Extended Abstracts · doi:10.1145/3491101.3516384 · Cited by 5 · see citing works
Tang, X., & Zhang, W. (2026) · Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3790464 · Cited by 3 · see citing works
- Authors
- Abdi, N., Zhan, N., Ramokapane, M., & Such, J.
- Affiliations
- Institut national de recherche en sciences et technologies du numérique (FR)
- Publisher
- ACM
- Published
- 2021-05-06
- OpenAlex
- W3160042174
- Cited by
- 1,240
- References
- 10
- Citations by year
- 2026: 1 · 2025: 10 · 2024: 399 · 2023: 273 · 2022: 241 · 2021: 127 · 2020: 29 · 2019: 8 · 2018: 6 · 2017: 8 · 2016: 4 · 2015: 7 · 2014: 1 · 2013: 4 · 2012: 2
- Type
- paratext
- Language
- en
- Open access
- green
- kclpure.kcl.ac.uk
- Licence
- cc-by
- Locations
- Research Portal (King's College London) (publishedVersion, cc-by)
- Funders
- Engineering and Physical Sciences Research Council (EP/R004471/1, EP/S027432/1)
- Topics
- Physical Sciences › Computer Science › Computer Science Applications › Online Learning and Analytics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Data Visualization and Analytics
- Physical Sciences › Computer Science › Information Systems › Recommender Systems and Techniques
- Keywords
- course selection, grade prediction, framing effects, workload, student effort, interactive visualization, data-driven decision support, performance prediction
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
